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AN AGILE NETWORK PROTOCOL FOR SECURE COMMUNICATIONS 
WITH ASSURED SYSTEM AVAILABILITY 

Related Applications 

This application claims priority from and bodily incorporates the subject matter of two 
previously filed provisional patent applications: serial number 60/106,261 (filed on October 30, 
1998) and serial number 60/137,704 (filed on June 7, 1999). 
Background of the Invention 

A tremendous variety of methods have been proposed and implemented to provide 
security and anonymity for communications over the Internet. The variety stems, in part, from the 
different needs of different Internet users. A basic heuristic framework to aid in discussing these 
different security techniques is illustrated in FIG. 1. Two terminals, an originating terminal 100 
and a destination terminal 110 are in communication over the Internet. It is desired for the 
communications to be secure, that is, immune to eavesdropping. For example, terminal 100 may 
transmit secret information to terminal 110 over the Internet 107. Also, it may be desired to 
prevent an eavesdropper from discovering that terminal 100 is in communication with terminal 
1 10. For example, if terminal 100 is a user and terminal 1 10 hosts a web site, terminal 100's user 
may not want anyone in the intervening networks to know what web sites he is "visiting." 
Anonymity would thus be an issue, for example, for companies that want to keep their market 
research interests private and thus would prefer to prevent outsiders from knowing which web- 
sites or other Internet resources they are "visiting." These two security issues may be called data 
security and anonymity, respectively. 

Data security is usually tackled using some form of data encryption. An encryption key 48 
is known at both the originating and terminating terminals 100 and 1 10. The keys may be private 
and public at the originating and destination terminals 100 and 110, respectively or they may be 
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symmetrical keys (the same key is used by both parties to encrypt and decrypt). Many encryption 
methods are known and usable in this context. 

To hide traffic from a local administrator or ISP, a user can employ a local proxy server in 
communicating over an encrypted channel with an outside proxy such that the local administrator 
or ISP only sees the encrypted traffic. Proxy servers prevent destination servers from determining 
the identities of the originating clients. This system employs an intermediate server interposed 
between client and destination server. The destination server sees only the Internet Protocol (IP) 
address of the proxy server and not the originating client. The target server only sees the address 
of the outside proxy. This scheme relies on a trusted outside proxy server. Also, proxy schemes 
are vulnerable to traffic analysis methods of determining identities of transmitters and receivers. 
Another important limitation of proxy servers is that the server knows the identities of both 
calling and called parties. In many instances, an originating terminal, such as terminal A, would 
prefer to keep its identity concealed from the proxy, for example, if the proxy server is provided 
by an Internet service provider (ISP). 

To defeat traffic analysis, a scheme called Chaum's mixes employs a proxy server that 
transmits and receives fixed length messages, including dummy messages. Multiple originating 
terminals are connected through a mix (a server) to multiple target servers. It is difficult to tell 
which of the originating terminals are communicating to which of the connected target servers, 
and the dummy messages confuse eavesdroppers' efforts to detect communicating pairs by 
analyzing traffic. A drawback is that there is a risk that the mix server could be compromised. One 
way to deal with this risk is to spread the trust among multiple mixes. If one mix is compromised, 
the identities of the originating and target terminals may remain concealed. This strategy requires 
a number of alternative mixes so that the intermediate servers interposed between the originating 
and target terminals are not determinable except by compromising more than one mix. The 
strategy wraps the message with multiple layers of encrypted addresses. The first mix in a 
sequence can decrypt only the outer layer of the message to reveal the next destination mix in 
sequence. The second mix can decrypt the message to reveal the next mix and so on. The target 
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server receives the message and, optionally, a multi-layer encrypted payload containing return 
information to send data back in the same fashion. The only way to defeat such a mix scheme is to 
collude among mixes. If the packets are all fixed-length and intermixed with dummy packets, 
there is no way to do any kind of traffic analysis. 

Still another anonymity technique, called 'crowds/ protects the identity of the originating 
terminal from the intermediate proxies by providing that originating terminals belong to groups of 
proxies called crowds. The crowd proxies are interposed between originating and target terminals. 
Each proxy through which the message is sent is randomly chosen by an upstream proxy. Each 
intermediate proxy can send the message either to another randomly chosen proxy in the "crowd" 
or to the destination. Thus, even crowd members cannot determine if a preceding proxy is the 
originator of the message or if it was simply passed from another proxy. 

ZKS (Zero-Knowledge Systems) Anonymous IP Protocol allows users to select up to any 
of five different pseudonyms, while desktop software encrypts outgoing traffic and wraps it in 
User Datagram Protocol (UDP) packets. The first server in a 2+-hop system gets the UDP 
packets, strips off one layer of encryption to add another, then sends the traffic to the next server, 
which strips off yet another layer of encryption and adds a new one. The user is permitted to 
control the number of hops. At the final server, traffic is decrypted with an untraceable IP address. 
The technique is called onion-routing. This method can be defeated using traffic analysis. For a 
simple example, bursts of packets from a user during low-duty periods can reveal the identities of 
sender and receiver. 

Firewalls attempt to protect LANs from unauthorized access and hostile exploitation or 
damage to computers connected to the LAN. Firewalls provide a server through which all access 
to the LAN must pass. Firewalls are centralized systems that require administrative overhead to 
maintain. They can be compromised by virtual-machine applications ("applets"). They instill a 
false sense of security that leads to security breaches for example by users sending sensitive 
information to servers outside the firewall or encouraging use of modems to sidestep the firewall 
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security. Firewalls are not useful for distributed systems such as business travelers, extranets, 
small teams, etc. 
Summary of the Invention 

A secure mechanism for communicating over the internet, including a protocol referred to 
as the Tunneled Agile Routing Protocol (TARP), uses a unique two-layer encryption format and 
special TARP routers. TARP routers are similar in function to regular IP routers. Each TARP 
router has one or more IP addresses and uses normal IP protocol to send IP packet messages 
("packets" or "datagrams"). The IP packets exchanged between TARP terminals via TARP 
routers are actually encrypted packets whose true destination address is concealed except to 
TARP routers and servers. The normal or "clear" or "outside" IP header attached to TARP IP 
packets contains only the address of a next hop router or destination server. That is, instead of 
indicating a final destination in the destination field of the IP header, the TARP packet's IP header 
always points to a next-hop in a series of TARP router hops, or to the final destination. This 
means there is no overt indication from an intercepted TARP packet of the true destination of the 
TARP packet since the destination could always be next-hop TARP router as well as the final 
destination. 

Each TARP packet's true destination is concealed behind a layer of encryption generated 
using a link key. The link key is the encryption key used for encrypted communication between 
the hops intervening between an originating TARP terminal and a destination TARP terminal. 
Each TARP router can remove the outer layer of encryption to reveal the destination router for 
each TARP packet. To identify the link key needed to decrypt the outer layer of encryption of a 
TARP packet, a receiving TARP or routing terminal may identify the transmitting terminal by the 
sender/receiver IP numbers in the cleartext IP header. 

Once the outer layer of encryption is removed, the TARP router determines the final 
destination. Each TARP packet 140 undergoes a minimum number of hops to help foil traffic 
analysis. The hops may be chosen at random or by a fixed value. As a result, each TARP packet 
may make random trips among a number of geographically disparate routers before reaching its 
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destination. Each trip is highly likely to be different for each packet composing a given message 
because each trip is independently randomly determined. This feature is called agile routing. The 
fact that different packets take different routes provides distinct advantages by making it difficult 
for an interloper to obtain all the packets forming an entire multi-packet message. The associated 
advantages have to do with the inner layer of encryption discussed below. Agile routing is 
combined with another feature that furthers this purpose; a feature that ensures that any message 
is broken into multiple packets. 

The IP address of a TARP router may not remain constant; a feature called IP agility. 
Each TARP router, independently or under direction from another TARP terminal or router, may 
change its IP address. A separate, unchangeable identifier or address is also defined. This address, 
called the TARP address, is known only to TARP routers and terminals and may be correlated at 
any time by a TARP router or a TARP terminal using a Lookup Table (LUT). When a TARP 
router or terminal changes its IP address, it updates the other TARP routers and terminals which 
in turn update their respective LUTs. 

The message payload is hidden behind an inner layer of encryption in the TARP packet 
that can only be unlocked using a session key. The session key is not available to any of the 
intervening TARP routers. The session key is used to decrypt the payloads of the TARP packets 
permitting the data stream to be reconstructed. 

Communication may be made private using link and session keys, which in turn may be 
shared and used according any desired method. For example, public /private keys or symmetric 
keys may be used. 

To transmit a data stream, a TARP originating terminal constructs a series of TARP 
packets from a series of IP packets generated by a network (IP) layer process. (Note that the 
terms "network layer," "data link layer," "application layer," etc. used in this specification 
correspond to the Open Systems Interconnection (OSI) network terminology.) The payloads of 
these packets are assembled into a block and chain-block encrypted using the session key. This 
assumes, of course, that all the IP packets are destined for the same TARP terminal. The block is 
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then interleaved and the interleaved encrypted block is broken into a series of payloads, one for 
each TARP packet to be generated. Special TARP headers IP T are then added to each payload 
using the IP headers from the data stream packets. The TARP headers can be identical to normal 
IP headers or customized in some way. They should contain a formula or data for deinterleaving 
5 the data at the destination TARP terminal, a time-to-live (TTL) parameter to indicate the number 
of hops still to be executed, a data type identifier which indicates whether the payload contains, 
for example, TCP or UDP data, the sender's TARP address, the destination TARP address, and 
an indicator as to whether the packet contains real or decoy data or a formula for filtering out 
decoy data if decoy data is spread in some way through the TARP payload data. 

10 Note that although chain-block encryption is discussed here with reference to the session 

key, any encryption method may be used. Preferably, as in chain block encryption, a method 
should be used that makes unauthorized decryption difficult without an entire result of the 
encryption process. Thus, by separating the encrypted block among multiple packets and making 
it difficult for an interloper to obtain access to all of such packets, the contents of the 

15 communications are provided an extra layer of security. 

Decoy or dummy data can be added to a stream to help foil traffic analysis by reducing the 
peak-to-average network load. It may be desirable to provide the TARP process with an ability to 
respond to the time of day or other criteria to generate more decoy data during low traffic periods 
so that communication bursts at one point in the Internet cannot be tied to communication bursts 

20 at another point to reveal the communicating endpoints. 

Dummy data also helps to break the data into a larger number of inconspicuously-sized 
packets permitting the interleave window size to be increased while maintaining a reasonable size 
for each packet. (The packet size can be a single standard size or selected from a fixed range of 
sizes.) One primary reason for desiring for each message to be broken into multiple packets is 

25 apparent if a chain block encryption scheme is used to form the first encryption layer prior to 
interleaving. A single block encryption may be applied to portion, or entirety, of a message, and 
that portion or entirety then interleaved into a number of separate packets. Considering the agile 
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IP routing of the packets, and the attendant difficulty of reconstructing an entire sequence of 
packets to form a single block-encrypted message element, decoy packets can significantly 
increase the difficulty of reconstructing an entire data stream. 

The above scheme may be implemented entirely by processes operating between the data 
link layer and the network layer of each server or terminal participating in the TARP system. 
Because the encryption system described above is insertable between the data link and network 
layers, the processes involved in supporting the encrypted communication may be completely 
transparent to processes at the IP (network) layer and above. The TARP processes may also be 
completely transparent to the data link layer processes as well. Thus, no operations at or above 
the Network layer, or at or below the data link layer, are affected by the insertion of the TARP 
stack. This provides additional security to all processes at or above the network layer, since the 
difficulty of unauthorized penetration of the network layer (by, for example, a hacker) is increased 
substantially. Even newly developed servers running at the session layer leave all processes below 
the session layer vulnerable to attack. Note that in this architecture, security is distributed. That is, 
notebook computers used by executives on the road, for example, can communicate over the 
Internet without any compromise in security. 

IP address changes made by TARP terminals and routers can be done at regular intervals, 
at random intervals, or upon detection of "attacks." The variation of IP addresses hinders traffic 
analysis that might reveal which computers are communicating, and also provides a degree of 
immunity from attack. The level of immunity from attack is roughly proportional to the rate at 
which the IP address of the host is changing. 

As mentioned, IP addresses may be changed in response to attacks. An attack may be 
revealed, for example, by a regular series of messages indicating that a router is being probed in 
some way. Upon detection of an attack, the TARP layer process may respond to this event by 
changing its IP address. In addition, it may create a subprocess that maintains the original IP 
address and continues interacting with the attacker in some manner. 
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Decoy packets may be generated by each TARP terminal on some basis determined by an 
algorithm. For example, the algorithm may be a random one which calls for the generation of a 
packet on a random basis when the terminal is idle. Alternatively, the algorithm may be responsive 
to time of day or detection of low traffic to generate more decoy packets during low traffic times. 
Note that packets are preferably generated in groups, rather than one by one, the groups being 
sized to simulate real messages. In addition, so that decoy packets may be inserted in normal 
TARP message streams, the background loop may have a latch that makes it more likely to insert 
decoy packets when a message stream is being received. Alternatively, if a large number of decoy 
packets is received along with regular TARP packets, the algorithm may increase the rate of 
dropping of decoy packets rather than forwarding them. The result of dropping and generating 
decoy packets in this way is to make the apparent incoming message size different from the 
apparent outgoing message size to help foil traffic analysis. 

In various other embodiments of the invention, a scalable version of the system may be 
constructed in which a plurality of IP addresses are preassigned to each pair of communicating 
nodes in the network. Each pair of nodes agrees upon an algorithm for "hopping" between IP 
addresses (both sending and receiving), such that an eavesdropper sees apparently continuously 
random IP address pairs (source and destination) for packets transmitted between the pair. 
Overlapping or "reusable" IP addresses may be allocated to different users on the same subnet, 
since each node merely verifies that a particular packet includes a valid source/destination pair 
from the agreed-upon algorithm. Source/destination pairs are preferably not reused between any 
two nodes during any given end-to-end session, though limited IP block sizes or lengthy sessions 
might require it. 

Brief Description of the Drawings 

FIG. 1 is an illustration of secure communications over the Internet according to a prior 
art embodiment. 

FIG. 2 is an illustration of secure communications over the Internet according to a an 
embodiment of the invention. 
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FIG. 3a is an illustration of a process of forming a tunneled IP packet according to an 
embodiment of the invention. 

FIG. 3b is an illustration of a process of forming a tunneled IP packet according to 
another embodiment of the invention. 

FIG. 4 is an illustration of an OSI layer location of processes that may be used to 
implement the invention. 

FIG. 5 is a flow chart illustrating a process for routing a tunneled packet according to an 
embodiment of the invention. 

FIG. 6 is a flow chart illustrating a process for forming a tunneled packet according to an 
embodiment of the invention. 

FIG. 7 is a flow chart illustrating a process for receiving a tunneled packet according to an 
embodiment of the invention. 

FIG. 8 shows how a secure session is established and synchronized between a client and a 
TARP router. 

FIG. 9 shows an IP address hopping scheme between a client computer and TARP router 
using transmit and receive tables in each computer. 

FIG. 10 shows physical link redundancy among three Internet Service Providers (ISPs) 
and a client computer. 

FIG. 1 1 shows how multiple IP packets can be embedded into a single "frame" such as an 
Ethernet frame, and further shows the use of a discriminator field to camouflage true packet 
recipients. 

FIG 12A shows a system that employs hopped hardware addresses, hopped IP addresses, 
and hopped discriminator fields. 

FIG. 12B shows several different approaches for hopping hardware addresses, IP 
addresses, and discriminator fields in combination. 

FIG. 13 shows a technique for automatically re-establishing synchronization between 
sender and receiver through the use of a partially public sync value. 
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FIG. 14 shows a "checkpoint" scheme for regaining synchronization between a sender and 
recipient. 

FIG. 15 shows further details of the checkpoint scheme of FIG. 14. 

FIG. 16 shows how two addresses can be decomposed into a plurality of segments for 
5 comparison with presence vectors. 

FIG. 17 shows a storage array for a receiver's active addresses. 

FIG. 18 shows the receiver's storage array after receiving a sync request. 

FIG. 19 shows the receiver's storage array after new addresses have been generated. 

FIG. 20 shows a system employing distributed transmission paths. 
10 FIG. 21 shows a plurality of link transmission tables that can be used to route packets in 

the system of FIG. 20. 
Detailed Description of the Embodiments 

Referring to FIG. 2, a secure mechanism for communicating over the internet employs a 
number of special routers or servers, called TARP routers 122-127 that are similar to regular IP 
15 routers 128-132 in that each has one or more IP addresses and uses normal IP protocol to send 
normal-looking IP packet messages, called TARP packets 140. TARP packets 140 are identical to 
normal IP packet messages that are routed by regular IP routers 128-132 because each TARP 
packet 140 contains a destination address as in a normal IP packet. However, instead of indicating 
a final destination in the destination field of the IP header, the TARP packet's 140 IP header 
20 always points to a next-hop in a series of TARP router hops, or the final destination, TARP 
terminal 110. Because the header of the TARP packet contains only the next-hop destination, 
there is no overt indication from an intercepted TARP packet of the true destination of the TARP 
packet 140 since the destination could always be the next-hop TARP router as well as the final 
destination, TARP terminal 110. 
25 Each TARP packet's true destination is concealed behind an outer layer of encryption 

generated using a link key 146. The link key 146 is the encryption key used for encrypted 
communication between the end points (TARP terminals or TARP routers) of a single link in the 
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chain of hops connecting the originating TARP terminal 100 and the destination TARP terminal 
1 10. Each TARP router 122-127, using the link key 146 it uses to communicate with the previous 
hop in a chain, can use the link key to reveal the true destination of a TARP packet. To identify 
the link key needed to decrypt the outer layer of encryption of a TARP packet, a receiving TARP 
or routing terminal may identify the transmitting tenninal (which may indicate the link key used) 
by the sender field of the clear TP header. Alternatively, this identity may be hidden behind another 
layer of encryption in available bits in the clear IP header. Each TARP router, upon receiving a 
TARP message, determines if the message is a TARP message by using authentication data in the 
TARP packet. This could be recorded in available bytes in the TARP packet's IP header. 
Alternatively, TARP packets could be authenticated by attempting to decrypt using the link key 
146 and determining if the results are as expected. The former may have computational 
advantages because it does not involve a decryption process. 

Once the outer layer of decryption is completed by a TARP router 122-127, the TARP 
router determines the final destination. The system is preferably designed to cause each TARP 
packet 140 to undergo a niinimum number of hops to help foil traffic analysis. The time to live 
counter in the IP header of the TARP message may be used to indicate a number of TARP router 
hops yet to be completed. Each TARP router then would decrement the counter and determine 
from that whether it should forward the TARP packet 140 to another TARP router 122-127 or to 
the destination TARP terminal 110. If the time to live counter is zero or below zero after 
decrementing, for an example of usage, the TARP router receiving the TARP packet 140 may 
forward the TARP packet 140 to the destination TARP terminal 1 10. If the time to live counter is 
above zero after decrementing, for an example of usage, the TARP router receiving the TARP 
packet 140 may forward the TARP packet 140 to a TARP router 122-127 that the current TARP 
terminal chooses at random. As a result, each TARP packet 140 is routed through some minimum 
number of hops of TARP routers 122-127 which are chosen at random. 

Thus, each TARP packet, irrespective of the traditional factors determining traffic in the 
Internet, makes random trips among a number of geographically disparate routers before reaching 
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its destination and each trip is highly likely to be different for each packet composing a given 
message because each trip is independently randomly determined as described above. This feature 
is called agile routing. For reasons that will become clear shortly, the fact that different packets 
take different routes provides distinct advantages by making it difficult for an interloper to obtain 
all the packets forming an entire multi-packet message. Agile routing is combined with another 
feature that furthers this purpose, a feature that ensures that any message is broken into multiple 
packets. 

A TARP router receives a TARP packet when an IP address used by the TARP router 
coincides with the IP address in the TARP packet's IP header IP C . The IP address of a TARP 
router, however, may not remain constant. To avoid and manage attacks, each TARP router, 
independently or under direction from another TARP terminal or router, may change its IP 
address. A separate, unchangeable identifier or address is also defined. This address, called the 
TARP address, is known only to TARP routers and terminals and may be correlated at any time 
by a TARP router or a TARP terminal using a Lookup Table (LUT). When a TARP router or 
terminal changes its IP address, it updates the other TARP routers and terminals which in turn 
update their respective LUTs. In reality, whenever a TARP router looks up the address of a 
destination in the encrypted header, it must convert a TARP address to a real IP address using its 
LUT. 

While every TARP router receiving a TARP packet has the ability to determine the 
packet's final destination, the message payload is embedded behind an inner layer of encryption in 
the TARP packet that can only be unlocked using a session key. The session key is not available 
to any of the TARP routers 122-127 intervening between the originating 100 and destination 1 10 
TARP terminals. The session key is used to decrypt the payloads of the TARP packets 140 
permitting an entire message to be reconstructed. 

In one embodiment, communication may be made private using link and session keys, 
which in turn may be shared and used according any desired method. For example, a public key or 
symmetric keys may be communicated between link or session endpoints using a public key 
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method. Any of a variety of other mechanisms for securing data to ensure that only authorized 
computers can have access to the private information in the TARP packets 140 may be used as 
desired. 

Referring to FIG. 3a, to construct a series of TARP packets, a data stream 300 of IP 
packets 207a, 207b, 207c, etc., such series of packets being formed by a network (IP) layer 
process,, is broken into a series of small sized segments. In the present example, equal-sized 
segments 1-9 are defined and used to construct a set of interleaved data packets A, B, and C. 
Here it is assumed that the number of interleaved packets A, B, and C formed is three and that the 
number of IP packets 207a-207c used to form the three interleaved packets A, B, and C is exactly 
three. Of course, the number of IP packets spread over a group of interleaved packets may be any 
convenient number as may be the number of interleaved packets over which the incoming data 
stream is spread. The latter, the number of interleaved packets over which the data stream is 
spread, is called the interleave window. 

To create a packet, the transmitting software interleaves the normal IP packets 207a et 
seq. to form a new set of interleaved payload data 320. This payload data 320 is then encrypted 
using a session key to form a set of session-key-encrypted payload data 330, each of which, A, B, 
and C, will form the payload of a TARP packet. Using the IP header data, from the original 
packets 207a-207c, new TARP headers IP T are formed. The TARP headers IP T can be identical to 
normal IP headers or customized in some way. In a preferred embodiment, the TARP headers n» T 
are IP headers with added data providing the following information required for routing and 
reconstruction of messages, some of which data is ordinarily, or capable of being, contained in 
normal IP headers: 

1. A window sequence number - an identifier that indicates where the packet 
belongs in the original message sequence. 

2. An interleave sequence number - an identifier that indicates the interleaving 
sequence used to form the packet so that the packet can be deinterleaved along with 
other packets in the interleave window. 
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3 . A time-to-live (TTL) datum - indicates the number of TARP-router-hops to be 
executed before the packet reaches its destination. Note that the TTL parameter may 
provide a datum to be used in a probabilistic formula for determining whether to route 
the packet to the destination or to another hop. 

4. Data type identifier - indicates whether the payload contains, for example, 
TCP or UDP data. 

5. Sender's address - indicates the sender's address in the TARP network. 

6. Destination address - indicates the destination terminal's address in the TARP 
network. 

7. Decoy/Real - an indicator of whether the packet contains real message data or 
dummy decoy data or a combination. 

Obviously, the packets going into a single interleave window must include only packets 
with a common destination. Thus, it is assumed in the depicted example that the IP headers of IP 
packets 207a-207c all contain the same destination address or at least will be received by the same 
terminal so that they can be deinterleaved. Note that dummy or decoy data or packets can be 
added to form a larger interleave window than would otherwise be required by the size of a given 
message. Decoy or dummy data can be added to a stream to help foil traffic analysis by leveling 
the load on the network. Thus, it may be desirable to provide the TARP process with an ability to 
respond to the time of day or other criteria to generate more decoy data during low traffic periods 
so that communication bursts at one point in the Internet cannot be tied to communication bursts 
at another point to reveal the communicating endpoints. 

Dummy data also helps to break the data into a larger number of inconspicuously-sized 
packets permitting the interleave window size to be increased while maintaining a reasonable size 
for each packet. (The packet size can be a single standard size or selected from a fixed range of 
sizes.) One primary reason for desiring for each message to be broken into multiple packets is 
apparent if a chain block encryption scheme is used to form the first encryption layer prior to 
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interleaving. A single block encryption may be applied to portion, or entirety, of a message, and 
that portion or entirety then interleaved into a number of separate packets. 

Referring to FIG. 3b, in an alternative mode of TARP packet construction, a series of IP 
packets are accumulated to make up a predefined interleave window. The payloads of the packets 
are used to construct a single block 520 for chain block encryption using the session key. The 
payloads used to form the block are presumed to be destined for the same terminal. The block size 
may coincide with the interleave window as depicted in the example embodiment of FIG. 3b. 
After encryption, the encrypted block is broken into separate payloads and segments which are 
interleaved as in the embodiment of Fig 3a. The resulting interleaved packets A, B, and C, are 
then packaged as TARP packets with TARP headers as in the Example of FIG. 3a. The remaining 
process is as shown in, and discussed with reference to, FIG. 3a. 

Once the TARP packets 340 are formed, each entire TARP packet 340, including the 
TARP header D? T , is encrypted using the link key for communication with the first-hop-TARP 
router. The first hop TARP router is randomly chosen. A final unencrypted IP header n» c is added 
to each encrypted TARP packet 340 to form a normal IP packet 360 that can be transmitted to a 
TARP router. Note that the process of constructing the TARP packet 360 does not have to be 
done in stages as described. The above description is just a useful heuristic for describing the final 
product, namely, the TARP packet. 

Note that, TARP header n> T could be a completely custom header configuration with no 
similarity to a normal IP header except that it contain the information identified above. This is so 
since this header is interpreted by only TARP routers. 

The above scheme may be implemented entirely by processes operating between the data 
link layer and the network layer of each server or terminal participating in the TARP system. 
Referring to FIG. 4, a TARP transceiver 405 can be an originating terminal 100, a destination 
terminal 110, or a TARP router 122-127. In each TARP Transceiver 405, a transmitting process 
is generated to receive normal packets from the Network (IP) layer and generate TARP packets 
for communication over the network. A receiving process is generated to receive normal IP 
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packets containing TARP packets and generate from these normal IP packets which are "passed 
up" to the Network (IP) layer. Note that where the TARP Transceiver 405 is a router, the 
received TARP packets 140 are not processed into a stream of IP packets 415 because they need 
only be authenticated as proper TARP packets and then passed to another TARP router or a 
TARP destination terminal 110. The intervening process, a "TARP Layer" 420, could be 
combined with either the data link layer 430 or the Network layer 410. In either case, it would 
intervene between the data link layer 430 so that the process would receive regular IP packets 
containing embedded TARP packets and "hand up" a series of reassembled IP packets to the 
Network layer 410. As an example of combining the TARP layer 420 with the data link layer 430, 
a program may augment the normal processes running a communications card, for example, an 
ethernet card. Alternatively, the TARP layer processes may form part of a dynamically loadable 
module that is loaded and executed to support communications between the network and data 
link layers. 

Because the encryption system described above can be inserted between the data link and 
network layers, the processes involved in supporting the encrypted communication may be 
completely transparent to processes at the IP (network) layer and above. The TARP processes 
may also be completely transparent to the data link layer processes as well. Thus, no operations at 
or above the network layer, or at or below the data link layer, are affected by the insertion of the 
TARP stack. This provides additional security to all processes at or above the network layer, 
since the difficulty of unauthorized penetration of the network layer (by, for example, a hacker) is 
increased substantially. Even newly developed servers running at the session layer leave all 
processes below the session layer vulnerable to attack. Note that in this architecture, security is 
distributed. That is, notebook computers used by executives on the road, for example, can 
communicate over the Internet without any compromise in security. 

Note that IP address changes made by TARP terminals and routers can be done at regular 
intervals, at random intervals, or upon detection of "attacks." The variation of IP addresses 
hinders traffic analysis that might reveal which computers are communicating, and also provides a 
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degree of immunity from attack.. The level of immunity from attack is roughly proportional to the 
rate at which the IP address of the host is changing. 

As mentioned, IP addresses may be changed in response to attacks. An attack may be 
revealed, for example, by a regular series of messages indicates that a router is being probed in 
some way. Upon detection of an attack, the TARP layer process may respond to this event by 
changing its IP address. To accomplish this, the TARP process will construct a TARP-formatted 
message, in the style of Internet Control Message Protocol (ICMP) datagrams as an example; 
this message will contain the machine's TARP address, its previous IP address, and its new IP 
address. The TARP layer will transmit this packet to at least one known TARP router; then upon 
receipt and validation of the message, the TARP router will update its LUT with the new IP 
address for the stated TARP address. The TARP router will then format a similar message, and 
broadcast it to the other TARP routers so that they may update their LUTs. Since the total 
number of TARP routers on any given subnet is expected to be relatively small, this process of 
updating the LUTs should be relatively fast. It may not, however, work as well when there is a 
relatively large number of TARP routers and/or a relatively large number of clients; this has 
motivated a refinement of this architecture to provide scalability; this refinement has led to a 
second embodiment, which is discussed below. 

Upon detection of an attack, the TARP process may also create a subprocess that 
maintains the original IP address and continues interacting with the attacker. The latter may 
provide an opportunity to trace the attacker or study the attacker's methods (called "fishbowling" 
drawing upon the analogy of a small fish in a fish bowl that "thinks" it is in the ocean but is 
actually under captive observation). A history of the communication between the attacker and the 
abandoned (fishbowled) IP address can be recorded or transmitted for human analysis or further 
synthesized for purposes of responding in some way. 

As mentioned above, decoy or dummy data or packets can be added to outgoing data 
streams by TARP terminals or routers. In addition to making it convenient to spread data over a 
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larger number of separate packets, such decoy packets can also help to level the load on inactive 
portions of the Internet to help foil traffic analysis efforts. 

Decoy packets may be generated by each TARP terminal 100, 1 10 or each router 122-127 
on some basis determined by an algorithm. For example, the algorithm may be a random one 
which calls for the generation of a packet on a random basis when the terminal is idle. 
Alternatively, the algorithm may be responsive to time of day or detection of low traffic to 
generate more decoy packets during low traffic times. Note that packets are preferably generated 
in groups, rather than one by one, the groups being sized to simulate real messages. In addition, 
so that decoy packets may be inserted in normal TARP message streams, the background loop 
may have a latch that makes it more likely to insert decoy packets when a message stream is being 
received. That is, when a series of messages are received, the decoy packet generation rate may 
be increased. Alternatively, if a large number of decoy packets is received along with regular 
TARP packets, the algorithm may increase the rate of dropping of decoy packets rather than 
forwarding them. The result of dropping and generating decoy packets in this way is to make the 
apparent incoming message size different from the apparent outgoing message size to help foil 
traffic analysis. The rate of reception of packets, decoy or otherwise, may be indicated to the 
decoy packet dropping and generating processes through perishable decoy and regular packet 
counters. (A perishable counter is one that resets or decrements its value in response to time so 
that it contains a high value when it is incremented in rapid succession and a small value when 
incremented either slowly or a small number of times in rapid succession.) Note that destination 
TARP terminal 1 10 may generate decoy packets equal in number and size to those TARP packets 
received to make it appear it is merely routing packets and is therefore not the destination 
terminal. 

Referring to FIG. 5, the following particular steps may be employed in the above- 
described method for routing TARP packets. 
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• SO. A background loop operation is performed which applies an algorithm which determines 
the generation of decoy IP packets. The loop is interrupted when an encrypted TARP packet 
is received. 

• S2. The TARP packet may be probed in some way to authenticate the packet before 
attempting to decrypt it using the link key. That is, the router may determine that the packet is 
an authentic TARP packet by performing a selected operation on some data included with the 
clear IP header attached to the encrypted TARP packet contained in the payload. This makes 
it possible to avoid performing decryption on packets that are not authentic TARP packets. 

• S3. The TARP packet is decrypted to expose the destination TARP address and an indication 
of whether the packet is a decoy packet or part of a real message. 

• S4. If the packet is a decoy packet, the perishable decoy counter is incremented. 

• S5. Based on the decoy generation/dropping algorithm and the perishable decoy counter 
value, if the packet is a decoy packet, the router may choose to throw it away. If the received 
packet is a decoy packet and it is determined that it should be thrown away (S6), control 
returns to step SO. 

• S7. The TTL parameter of the TARP header is decremented and it is determined if the TTL 
parameter is greater than zero. 

• S8. If the TTL parameter is greater than zero, a TARP address is randomly chosen from a list 
of TARP addresses maintained by the router and the link key and IP address corresponding to 
that TARP address memorized for use in creating a new IP packet containing the TARP 
packet. 

• S9. If the TTL parameter is zero or less, the link key and IP address corresponding to the 
TARP address of the destination are memorized for use in creating the new IP packet 
containing the TARP packet. 

• S10. The TARP packet is encrypted using the memorized link key. 
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• SI 1. An IP header is added to the packet that contains the stored IP address, the encrypted 
TARP packet wrapped with an IP header, and the completed packet transmitted to the next 
hop or destination. 

Referring to FIG. 6, the following particular steps may be employed in the above- 
described method for generating TARP packets. 

• S20. A background loop operation applies an algorithm that determines the generation of 
decoy IP packets. The loop is interrupted when a data stream containing IP packets is 
received for transmission. 

• S21. The received IP packets are grouped into a set consisting of messages with a constant IP 
destination address. The set is further broken down to coincide with a maximum size of an 
interleave window The set is encrypted, and interleaved into a set of payloads destined to 
become TARP packets. 

522. The TARP address corresponding to the IP address is determined from a lookup table 
and stored to generate the TARP header. An initial TTL count is generated and stored in the 
header. The TTL count may be random with minimum and maximum values or it may be fixed 
or determined by some other parameter. 

523. The window sequence numbers and interleave sequence numbers are recorded in the 
TARP headers of each packet. 

524. One TARP router address is randomly chosen for each TARP packet and the IP address 
corresponding to it stored for use in the clear IP header. The link key corresponding to this 
router is identified and used to encrypt TARP packets containing interleaved and encrypted 
data and TARP headers. 

525. A clear IP header with the first hop router's real IP address is generated and added to 
each of the encrypted TARP packets and the resulting packets. 
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Referring to FIG. 7, the following particular steps may be employed in the above- 
described method for receiving TARP packets. 

• S40. A background loop operation is performed which applies an algorithm which determines 
5 the generation of decoy IP packets. The loop is interrupted when an encrypted TARP packet 

is received. 

• S42. The TARP packet may be probed to authenticate the packet before attempting to 
decrypt it using the link key. 

• S43. The TARP packet is decrypted with the appropriate link key to expose the destination 
10 TARP address and an indication of whether the packet is a decoy packet or part of a real 

message. 

• S44. If the packet is a decoy packet, the perishable decoy counter is incremented. 

• S45. Based on the decoy generation/dropping algorithm and the perishable decoy counter 
value, if the packet is a decoy packet, the receiver may choose to throw it away. 

15 • S46. The TARP packets are cached until all packets forming an interleave window are 
received. 

• S47. Once all packets of an interleave window are received, the packets are deinterleaved. 

• S48. The packets block of combined packets defining the interleave window is then decrypted 
using the session key. 

20 • S49. The decrypted block is then divided using the window sequence data and the IP T headers 
are converted into normal IPc headers. The window sequence numbers are integrated in the 
IPc headers. 

• S50. The packets are then handed up to the IP layer processes. 
Scalability Enhancements 

25 The IP agility feature described above relies on the ability to transmit IP address changes 

to all TARP routers. The embodiments including this feature will be referred to as "boutique" 
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embodiments due to potential limitations in scaling these features up for a large network, such as 
the Internet. (The "boutique" embodiments would, however, be robust for use in smaller 
networks, such as small virtual private networks, for example). One problem with the boutique 
embodiments is that if IP address changes are to occur frequently, the message traffic required to 
update all routers sufficiently quickly creates a serious burden on the Internet when the TARP 
router and/or client population gets large. The bandwidth burden added to the networks, for 
example in ICMP packets, that would be used to update all the TARP routers could overwhelm 
the Internet for a large scale implementation that approached the scale of the Internet. In other 
words, the boutique system's scalability is limited. 

A system can be constructed which trades some of the features of the above embodiments 
to provide the benefits of IP agility without the additional messaging burden. This is accomplished 
by IP address-hopping according to shared algorithms that govern IP addresses used between 
links participating in communications sessions between nodes such as TARP nodes. (Note that the 
IP hopping technique is also applicable to the boutique embodiment.) The IP agility feature 
discussed with respect to the boutique system can be modified so that it becomes decentralized 
under this scalable regime and governed by the above-described shared algorithm. Other features 
of the boutique system may be combined with this new type of IP-agility. 

The new embodiment has the advantage of providing IP agility governed by a local 
algorithm and set of IP addresses exchanged by each communicating pair of nodes. This local 
governance is session-independent in that it may govern communications between a pair of nodes, 
irrespective of the session or end points being transferred between the directly communicating 
pair of nodes. 

In the scalable embodiments, blocks of IP addresses are allocated to each node in the 
network. (This scalability will increase in the future, when Internet Protocol addresses are 
increased to 128-bit fields, vastly increasing the number of distinctly addressable nodes). Each 
node can thus use any of the IP addresses assigned to that node to communicate with other nodes 
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in the network. Indeed, each pair of communicating nodes can use a plurality of source IP 
addresses and destination IP addresses for communicating with each other. 

Each communicating pair of nodes in a chain participating in any session stores two blocks 
of IP addresses, called netblocks, and an algorithm and randomization seed for selecting, from 
each netblock, the next pair of source/destination IP addresses that will be used to transmit the 
next message. In other words, the algorithm governs the sequential selection of IP-address pairs, 
one sender and one receiver IP address, from each netblock. The combination of algorithm, seed, 
and netblock (IP address block) will be called a "hopblock." A router issues separate transmit and 
receive hopblocks to its clients. The send address and the receive address of the IP header of each 
outgoing packet sent by the client are filled with the send and receive IP addresses generated by 
the algorithm. The algorithm is "clocked 5 * (indexed) by a counter so that each time a pair is used, 
the algorithm turns out a new transmit pair for the next packet to be sent. 

The router's receive hopblock is identical to the client's transmit hopblock. The router 
uses the receive hopblock to predict what the send and receive IP address pair for the next 
expected packet from that client will be. Since packets can be received out of order, it is not 
possible for the router to predict with certainty what IP address pair will be on the next sequential 
packet. To account for this problem, the router generates a range of predictions encompassing 
the number of possible transmitted packet send/receive addresses, of which the next packet 
received could leap ahead. Thus, if there is a vanishingly small probability that a given packet will 
arrive at the router ahead of 5 packets transmitted by the client before the given packet, then the 
router can generate a series of 6 send/receive IP address pairs (or "hop window") to compare 
with the next received packet. When a packet is received, it is marked in the hop window as such, 
so that a second packet with the same IP address pair will be discarded. If an out-of-sequence 
packet does not arrive within a predetermined timeout period, it can be requested for 
retransmission or simply discarded from the receive table, depending upon the protocol in use for 
that communications session, or possibly by convention. 
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When the router receives the client's packet, it compares the send and receive IP 
addresses of the packet with the next N predicted send and receive IP address pairs and rejects the 
packet if it is not a member of this set. Received packets that do not have the predicted 
source/destination IP addresses falling with the window are rejected, thus thwarting possible 
hackers. (With the number of possible combinations, even a fairly large window would be hard to 
fall into at random.) If it is a member of this set, the router accepts the packet and processes it 
further. This link-based IP-hopping strategy, referred to as "IHOP," is a network element that 
stands on its own and is not necessarily accompanied by elements of the boutique system 
described above. If the routing agility feature described in connection with the boutique 
embodiment is combined with this link-based IP-hopping strategy, the router's next step would be 
to decrypt the TARP header to determine the destination TARP router for the packet and 
determine what should be the next hop for the packet. The TARP router would then forward the 
packet to a random TARP router or the destination TARP router with which the source TARP 
router has a link-based IP hopping communication established. 

Figure 8 shows how a client computer 801 and a TARP router 811 can establish a secure 
session. When client 801 seeks to establish an HOP session with TARP router 811, the client 
801 sends "secure synchronization" request ("SSYN") packet 821 to the TARP router 811. This 
SYN packet 821 contains the client's 801 authentication token, and may be sent to the router 811 
in an encrypted format The source and destination IP numbers on the packet 821 are the client's 
801 current fixed IP address, and a "known" fixed IP address for the router 811. (For security 
purposes, it may be desirable to reject any packets from outside of the local network that are 
destined for the router's known fixed IP address.) Upon receipt and validation of the client's 801 
SSYN packet 821, the router 811 respond by sending an encrypted "secure synchronization 
acknowledgment" ("SSYN ACK") 822 to the client 801. This SSYN ACK 822 will contain the 
transmit and receive hopblocks that the client 801 will use when communicating with the TARP 
router 811. The client 801 will acknowledge the TARP router's 811 response packet 822 by 
generating an encrypted SSYN ACK ACK packet 823 which will be sent from the client's 801 
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fixed IP address and to the TARP router's 811 known fixed IP address. The client 801 will 
simultaneously generate a SSYN ACK ACK packet; this SSYN ACK packet, referred to as the 
Secure Session Initiation (SSI) packet 824, will be sent with the first {sender, receiver} IP pair in 
the client's transmit table 921 (FIG. 9), as specified in the transmit hopblock provided by the 
5 TARP router 8 1 1 in the SSYN ACK packet 822. The TARP router 8 1 1 will respond to the SSI 
packet 824 with an SSI ACK packet 825, which will be sent with the first {sender, receiver} IP 
pair in the TARP router's transmit table 923. Once these packets have been successfully 
exchanged, the secure communications session is established, and all further secure 
communications between the client 801 and the TARP router 811 will be conducted via this 
10 secure session, as long as synchronization is maintained. If synchronization is lost, then the client 
801 and TARP router 802 may re-establish the secure session by the procedure outlined in Figure 
=F 8 and described above. 

yg While the secure session is active, both the client 901 and TARP router 91 1 (FIG. 9) will 

y maintain their respective transmit tables 921, 923 and receive tables 922, 924, as provided by the 

to 15 TARP router during session synchronization 822. It is important that the sequence of IP pairs in 
y* the client's transmit table 921 be identical to those in the TARP router's receive table 924; 

similarly, the sequence of IP pairs in the client's receive table 922 must be identical to those in the 
* router's transmit table 923. This is required for the session synchronization to be maintained. The 

J} client 901 need maintain only one transmit table 921 and one receive table 922 during the course 

20 of the secure session. Each sequential packet sent by the client 901 will employ the next {send, 
receive} IP address pair in the transmit table, regardless of TCP or UDP session. The TARP 
router 911 will expect each packet arriving from the client 901 to bear the next IP address pair 
shown in its receive table. 

Since packets can arrive out of order, however, the router 911 can maintain a "look 
25 ahead" buffer in its receive table, and will mark previously-received IP pairs as invalid for future 
packets; any future packet containing an IP pair that is in the look-ahead buffer but is marked as 
previously received will be discarded. Communications from the TARP router 911 to the client 
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901 are maintained in an identical manner; in particular, the router 911 will select the next IP 
address pair from its transmit table 923 when constructing a packet to send to the client 901, and 
the client 901 will maintain a look-ahead buffer of expected IP pairs on packets that it is receiving. 
Each TARP router will maintain separate pairs of transmit and receive tables for each client that is 
currently engaged in a secure session with or through that TARP router. 

While clients receive their hopblocks from the first server linking them to the Internet, 
routers exchange hopblocks. When a router establishes a link-based IP-hopping communication 
regime with another router, each router of the pair exchanges its transmit hopblock. The transmit 
hopblock of each router becomes the receive hopblock of the other router. The communication 
between routers is governed as described by the example of a client sending a packet to the first 
router. 

While the above strategy works fine in the IP milieu, many local networks that are 
connected to the Internet are ethernet systems. In ethernet, the IP addresses of the destination 
devices must be translated into hardware addresses, and vice versa, using known processes 
("address resolution protocol," and "reverse address resolution protocol"). However, if the link- 
based IP-hopping strategy is employed, the correlation process would become explosive and 
burdensome. An alternative to the link-based IP hopping strategy may be employed within an 
ethernet network. The solution is to provide that the node linking the Internet to the ethernet (call 
it the border node) use the link-based IP-hopping communication regime to communicate with 
nodes outside the ethernet LAN. Within the ethernet LAN, each TARP node would have a single 
IP address which would be addressed in the conventional way. Instead of comparing the {sender, 
receiver} IP address pairs to authenticate a packet, the intra-LAN TARP node would use one of 
the IP header extension fields to do so. Thus, the border node uses an algorithm shared by the 
intra-LAN TARP node to generate a symbol that is stored in the free field in the IP header, and 
the intra-LAN TARP node generates a range of symbols based on its prediction of the next 
expected packet to be received from that particular source IP address. The packet is rejected if it 
does not fall into the set of predicted symbols (for example, numerical values) or is accepted if it 
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does. Communications from the intra-LAN TARP node to the border node are accomplished in 
the same manner, though the algorithm will necessarily be different for security reasons. Thus, 
each of the communicating nodes will generate transmit and receive tables in a similar manner to 
that of Figure 9; the intra-LAN TARP nodes transmit table will be identical to the border node's 
receive table, and the intra-LAN TARP node's receive table will be identical to the border node's 
transmit table. 

The algorithm used for IP address-hopping can be any desired algorithm. For example, the 
algorithm can be a given pseudo-random number generator that generates numbers of the range 
covering the allowed IP addresses with a given seed. Alternatively, the session participants can 
assume a certain type of algorithm and specify simply a parameter for applying the algorithm. For 
example the assumed algorithm could be a particular pseudo-random number generator and the 
session participants could simply exchange seed values. 

Note that there is no permanent physical distinction between the originating and 
destination terminal nodes. Either device at either end point can initiate a synchronization of the 
pair. Note also that the authentication/synchronization-request (and acknowledgment) and 
hopblock-exchange may all be served by a single message so that separate message exchanges 
may not be required. 

As another extension to the stated architecture, multiple physical paths can be used by a 
client, in order to provide link redundancy and farther thwart attempts at denial of service and 
traffic monitoring. As shown in Figure 10, for example, client 1001 can establish three 
simultaneous sessions with each of three TARP routers provided by different ISPs 1011, 1012, 
1013. As an example, the client 1001 can use three different telephone lines 1021, 1022, 1023 to 
connect to the ISPs, or two telephone lines and a cable modem, etc. In this scheme, transmitted 
packets will be sent in a random fashion among the different physical paths. This architecture 
provides a high degree of communications redundancy, with improved immunity from denial-of- 
service attacks and traffic monitoring. 
FURTHER EXTENSIONS 
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The following describes various extensions to the techniques, systems, and methods 
described above. As described above, the security of communications occurring between 
computers in a computer network (such as the Internet, an Ethernet, or others) can be enhanced 
by using seemingly random source and destination Internet Protocol (IP) addresses for data 
5 packets transmitted over the network. This feature prevents eavesdroppers from determining 
which computers in the network are communicating with each other while permitting the two 
communicating computers to easily recognize whether a given received data packet is legitimate 
or not In one embodiment of the above-described systems, an IP header extension field is used 
to authenticate incoming packets on an Ethernet. 

10 Various extensions to the previously described techniques described herein include: (1) 

use of hopped hardware or "MAC" addresses in broadcast type network; (2) a self- 
synchronization technique that permits a computer to automatically regain synchronization with a 
sender; (3) synchronization algorithms that allow transmitting and receiving computers to quickly 
re-establish synchronization in the event of lost packets or other events; and (4) a fast-packet 

15 rejection mechanism for rejecting invalid packets. Any or all of these extensions can be combined 
with the features described above in any of various ways. 
A. Hardware Address Hopping 

Internet protocol-based communications techniques on a LAN — or across any dedicated 
physical medium— typically embed the IP packets within lower-level packets, often referred to as 

20 "frames." As shown in FIG. 11, for example, a first Ethernet frame 1150 comprises a frame 
header 1101 and two embedded IP packets JPl and IP2, while a second Ethernet frame 1160 
comprises a different frame header 1 104 and a single IP packet IP3. Each frame header generally 
includes a source hardware address 1 101 A and a destination hardware address 1 101B; other well- 
known fields in frame headers are omitted from FIG. 11 for clarity. Two hardware nodes 

25 communicating over a physical communication channel insert appropriate source and destination 
hardware addresses to indicate which nodes on the channel or network should receive the frame. 
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It may be possible for a nefarious listener to acquire information about the contents of a 
frame and/or its communicants by examining frames on a local network rather than (or in addition 
to) the IP packets themselves. This is especially true in broadcast media, such as Ethernet, where 
it is necessary to insert into the frame header the hardware address of the machine that generated 
the frame and the hardware address of the machine to which frame is being sent. All nodes on the 
network can potentially "see" all packets transmitted across the network. This can be a problem 
for secure communications, especially in cases where the communicants do not want for any third 
party to be able to identify who is engaging in the information exchange. One way to address this 
problem is to push the address-hopping scheme down to the hardware layer. In accordance with 
various embodiments of the invention, hardware addresses are "hopped" in a manner similar to 
that used to change IP addresses, such that a listener cannot determine which hardware node 
generated a particular message nor which node is the intended recipient. 

FIG. 12A shows a system in which Media Access Control ("MAC") hardware addresses 
are "hopped" in order to increase security over a network such as an Ethernet. While the 
description refers to the exemplary case of an Ethernet environment, the inventive principles are 
equally applicable to other types of communications media. In the Ethernet case, the MAC 
address of the sender and receiver are inserted into the Ethernet frame and can be observed by 
anyone on the LAN who is within the broadcast range for that frame. For secure communications, 
it becomes desirable to generate frames with MAC addresses that are not attributable to any 
specific sender or receiver. 

As shown in FIG. 12A, two computer nodes 1201 and 1202 communicate over a 
communication channel such as an Ethernet. Each node executes one or more application 
programs 1203 and 1218 that communicate by transmitting packets through communication 
software 1204 and 1217, respectively. Examples of application programs include video 
conferencing, e-mail, word processing programs, telephony, and the like. Communication 
software 1204 and 1217 can comprise, for example, an OSI layered architecture or "stack" that 
standardizes various services provided at different levels of functionality. 
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The lowest levels of communication software 1204 and 1217 communicate with hardware 
components 1206 and 1214 respectively, each of which can include one or more registers 1207 
and 1215 that allow the hardware to be reconfigured or controlled in accordance with various 
communication protocols. The hardware components (an Ethernet network interface card, for 
example) communicate with each other over the communication medium. Each hardware 
component is typically pre-assigned a fixed hardware address or MAC number that identifies the 
hardware component to other nodes on the network. One or more interface drivers control the 
operation of each card and can, for example, be configured to accept or reject packets from 
certain hardware addresses. As will be described in more detail below, various embodiments of 
the inventive principles provide for "hopping" different addresses using one or more algorithms 
and one or more moving windows that track a range of valid addresses to validate received 
packets. Packets transmitted according to one or more of the inventive principles will be 
generally referred to as "secure" packets or "secure communications" to differentiate them from 
ordinary data packets that are transmitted in the clear using ordinary, machine-correlated 
addresses. 

One straightforward method of generating non-attributable MAC addresses is an extension 
of the IP hopping scheme. In this scenario, two machines on the same LAN that desire to 
communicate in a secure fashion exchange random-number generators and seeds, and create 
sequences of quasi-random MAC addresses for synchronized hopping. The implementation and 
synchronization issues are then similar to that of IP hopping. 

This approach, however, runs the risk of using MAC addresses that are currently active on 
the LAN — which, in turn, could interrupt communications for those machines. Since an Ethernet 
MAC address is at present 48 bits in length, the chance of randomly misusing an active MAC 
address is actually quite small. However, if that figure is multiplied by a large number of nodes (as 
would be found on an extensive LAN), by a large number of frames (as might be the case with 
packet voice or streaming video), and by a large number of concurrent Virtual Private Networks 
(VPNs), then the chance that a non-secure machine's MAC address could be used in an address- 
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hopped frame can become non-trivial. In short, any scheme that runs even a small risk of 
interrupting communications for other machines on the LAN is bound to receive resistance from 
prospective system administrators. Nevertheless, it is technically feasible, and can be implemented 
without risk on a LAN on which there is a small number of machines, or if all of the machines on 
the LAN are engaging in MAC-hopped communications. 

Synchronized MAC address hopping may incur some overhead in the course of session 
establishment, especially if there are multiple sessions or multiple nodes involved in the 
communications. A simpler method of randomizing MAC addresses is to allow each node to 
receive and process every incident frame on the network. Typically, each network interface driver 
will check the destination MAC address in the header of every incident frame to see if it matches 
that machine's MAC address; if there is no match, then the frame is discarded. In one 
embodiment, however, these checks can be disabled, and every incident packet is passed to the 
TARP stack for processing. This will be referred to as "promiscuous" mode, since every incident 
frame is processed. Promiscuous mode allows the sender to use completely random, 
unsynchronized MAC addresses, since the destination machine is guaranteed to process the frame. 
The decision as to whether the packet was truly intended for that machine is handled by the TARP 
stack, which checks the source and destination IP addresses for a match in its TP synchronization 
tables. If no match is found, the packet is discarded; if there is a match, the packet is unwrapped, 
the inner header is evaluated, and if the inner header indicates that the packet is destined for that 
machine then the packet is forwarded to the IP stack — otherwise it is discarded. 

One disadvantage of purely-random MAC address hopping is its impact on processing 
overhead; that is, since every incident frame must be processed, the machine's CPU is engaged 
considerably more often than if the network interface driver is discriminating and rejecting packets 
unilaterally. A compromise approach is to select either a single fixed MAC address or a small 
number of MAC addresses (e.g., one for each virtual private network on an Ethernet) to use for 
MAC-hopped communications, regardless of the actual recipient for which the message is 
intended. In this mode, the network interface driver can check each incident frame against one (or 
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a few) pre-established MAC addresses, thereby freeing the CPU from the task of physical-layer 
packet discrimination. This scheme does not betray any useful information to an interloper on the 
LAN; in particular, every secure packet can already be identified by a unique packet type in the 
outer header. However, since all machines engaged in secure communications would either be 
5 using the same MAC address, or be selecting from a small pool of predetermined MAC addresses, 
the association between a specific machine and a specific MAC address is effectively broken. 

In this scheme, the CPU will be engaged more often than it would be in non-secure 
communications (or in synchronized MAC address hopping), since the network interface driver 
cannot always unilaterally discriminate between secure packets that are destined for that machine, 

10 and secure packets from other VPNs. However, the non-secure traffic is easily eliminated at the 
network interface, thereby reducing the amount of processing required of the CPU. There are 
boundary conditions where these statements would not hold, of course— e.g., if all of the traffic 
on the LAN is secure traffic, then the CPU would be engaged to the same degree as it is in the 
purely-random address hopping case; alternatively, if each VPN on the LAN uses a different 

15 MAC address, then the network interface can perfectly discriminate secure frames destined for the 
local machine from those constituting other VPNs. These are engineering tradeoffs that might be 
best handled by providing administrative options for the users when installing the software and/or 
establishing VPNs. 

Even in this scenario, however, there still remains a slight risk of selecting MAC addresses 
20 that are being used by one or more nodes on the LAN. One solution to this problem is to formally 
assign one address or a range of addresses for use in MAC-hopped communications. This is 
typically done via an assigned numbers registration authority; e.g., in the case of Ethernet, MAC 
address ranges are assigned to vendors by the Institute of Electrical and Electronics Engineers 
(IEEE). A formally-assigned range of addresses would ensure that secure frames do not conflict 
25 with any properly-configured and properly-fiinctioning machines on the LAN. 

Reference will now be made to FIGS. 12A and 12B in order to describe the many 
combinations and features that follow the inventive principles. As explained above, two computer 
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nodes 1201 and 1202 are assumed to be communicating over a network or communication 
medium such as an Ethernet. A communication protocol in each node (1204 and 1217, 
respectively) contains a modified element 1205 and 1216 that performs certain functions that 
deviate from the standard communication protocols. In particular, computer node 1201 
implements a first "hop" algorithm 1208X that selects seemingly random source and destination 
IP addresses (and, in one embodiment, seemingly random IP header discriminator fields) in order 
to transmit each packet to the other computer node. For example, node 1201 maintains a transmit 
table 1208 containing triplets of source (S), destination (D), and discriminator fields (DS) that are 
inserted into outgoing IP packet headers. The table is generated through the use of an 
appropriate algorithm (e.g., a random number generator that is seeded with an appropriate seed) 
that is known to the recipient node 1202. As each new IP packet is formed, the next sequential 
entry out of the sender's transmit table 1208 is used to populate the IP source, IP destination, and 
IP header extension field (e.g., discriminator field). It will be appreciated that the transmit table 
need not be created in advance but could instead be created on-the-fly by executing the algorithm 
when each packet is formed. 

At the receiving node 1202, the same IP hop algorithm 1222X is maintained and used to 
generate a receive table 1222 that lists valid triplets of source IP address, destination TP address, 
and discriminator field. This is shown by virtue of the first five entries of transmit table 1208 
matching the second five entries of receive table 1222. (The tables may be slightly offset at any 
particular time due to lost packets, misordered packets, or transmission delays). Additionally, 
node 1202 maintains a receive window W3 that represents a list of valid IP source, IP destination, 
and discriminator fields that will be accepted when received as part of an incoming IP packet. As 
packets are received, window W3 slides down the list of valid entries, such that the possible valid 
entries change over time. Two packets that arrive out of order but are nevertheless matched to 
entries within window W3 will be accepted; those falling outside of window W3 will be rejected 
as invalid. The length of window W3 can be adjusted as necessary to reflect network delays or 
other factors. 
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Node 1202 maintains a similar transmit table 1221 for creating IP packets and frames 
destined for node 1201 using a potentially different hopping algorithm 1221X, and node 1201 
maintains a matching receive table 1209 using the same algorithm 1209X. As node 1202 
transmits packets to node 1201 using seemingly random IP source, IP destination, and/or 
discriminator fields, node 1201 matches the incoming packet values to those falling within 
window Wl maintained in its receive table. In effect, transmit table 1208 of node 1201 is 
synchronized (i.e., entries are selected in the same order) to receive table 1222 of receiving node 
1202, Similarly, transmit table 1221 of node 1202 is synchronized to receive table 1209 of node 
1201. It will be appreciated that although a common algorithm is shown for the source, 
destination and discriminator fields in FIG, 12A (using, e.g., a different seed for each of the three 
fields), an entirely different algorithm could in fact be used to establish values for each of these 
fields. It will also be appreciated that one or two of the fields can be "hopped" rather than all 
three as illustrated. 

In accordance with another aspect of the invention, hardware or "MAC* addresses are 
hopped instead of or in addition to IP addresses and/or the discriminator field in order to improve 
security in a local area or broadcast-type network. To that end, node 1201 further maintains a 
transmit table 1210 using a transmit algorithm 1210X to generate source and destination 
hardware addresses that are inserted into frame headers (e.g., fields 1 101 A and 1 101B in FIG. 1 1) 
that are synchronized to a corresponding receive table 1224 at node 1202. Similarly, node 1202 
maintains a different transmit table 1223 containing source and destination hardware addresses 
that is synchronized with a corresponding receive table 1211 at node 1201. In this manner, 
outgoing hardware frames appear to be originating from and going to completely random nodes 
on the network, even though each recipient can determine whether a given packet is intended for 
it or not. It will be appreciated that the hardware hopping feature can be implemented at a 
different level in the communications protocol than the IP hopping feature (e.g., in a card driver 
or in a hardware card itself to improve performance). 
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FIG. 12B shows three different embodiments or modes that can be employed using the 
aforementioned principles. In a first mode referred to as "promiscuous" mode, a common 
hardware address (e.g., a fixed address for source and another for destination) or else a 
completely random hardware address is used by all nodes on the network, such that a particular 

5 packet cannot be attributed to any one node. Each node must initially accept all packets 
containing the common (or random) hardware address and inspect the IP addresses or 
discriminator field to determine whether the packet is intended for that node. In this regard, 
either the IP addresses or the discriminator field or both can be varied in accordance with an 
algorithm as described above. As explained previously, this may increase each node's overhead 

10 since additional processing is involved to determine whether a given packet has valid source and 
destination hardware addresses. 

In a second mode referred to as "promiscuous per VPN" mode, a small set of fixed 
hardware addresses are used, with a fixed source/destination hardware address used for all nodes 
communicating over a virtual private network. For example, if there are six nodes on an Ethernet, 

15 and the network is to be split up into two private virtual networks such that nodes on one VPN 
can communicate with only the other two nodes on its own VPN, then two sets of hardware 
addresses could be used: one set for the first VPN and a second set for the second VPN. This 
would reduce the amount of overhead involved in checking for valid frames since only packets 
arriving from the designated VPN would need to be checked. IP addresses and one or more 

20 discriminator fields could still be hopped as before for secure communication within the VPN. Of 
course, this solution compromises the anonymity of the VPNs (i.e., an outsider can easily tell 
what traffic belongs in which VPN, though he cannot correlate it to a specific machine/person). It 
also requires the use of a discriminator field to mitigate the vulnerability to certain types of DoS 
attacks. (For example, without the discriminator field, an attacker on the LAN could stream 

25 frames containing the MAC addresses being used by the VPN; rejecting those frames could lead 
to excessive processing overhead. The discriminator field would provide a low-overhead means of 
rejecting the false packets.) 



35 



0479.84602 



PATENT 



In a third mode referred to as "hardware hopping" mode, hardware addresses are varied as 
illustrated in FIG. 12A, such that hardware source and destination addresses are changed 
constantly in order to provide non-attributable addressing. Variations on these embodiments are 
of course possible, and the invention is not intended to be limited in any respect by these 

5 illustrative examples. 

B. Extending the Address Space 

Address hopping provides security and privacy. However, the level of protection is limited 
by the number of addresses in the blocks being hopped. A hopblock denotes a field or fields 
modulated on a packet-wise basis for the purpose of providing a VPN. For instance, if two nodes 

10 communicate with IP address hopping using hopblocks of 4 addresses (2 bits) each, there would 
be 16 possible address-pair combinations. A window of size 16 would result in most address pairs 
being accepted as valid most of the time. This limitation can be overcome by using a 
discriminator field in addition to or instead of the hopped address fields. The discriminator field 
would be hopped in exactly the same fashion as the address fields and it would be used to 

15 determine whether a packet should be processed by a receiver. 

Suppose that two clients, each using four-bit hopblocks, would like the same level of 
protection afforded to clients communicating via IP hopping between two A blocks (24 address 
bits eligible for hopping). A discriminator field of 20 bits, used in conjunction with the 4 address 
bits eligible for hopping in the IP address field, provides this level of protection. A 24-bit 

20 discriminator field would provide a similar level of protection if the address fields were not 
hopped or ignored. Using a discriminator field offers the following advantages: (1) an arbitrarily 
high level of protection can be provided, and (2) address hopping is unnecessary to provide 
protection. This may be important in environments where address hopping would cause routing 
problems. 

25 C. Synchronization Techniques 

It is generally assumed that once a sending node and receiving node have exchanged 
algorithms and seeds (or similar information sufficient to generate quasi-random source and 
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destination tables), subsequent communication between the two nodes will proceed smoothly. 
Realistically, however, two nodes may lose synchronization due to network delays or outages, or 
other problems. Consequently, it is desirable to provide means for re-establishing synchronization 
between nodes in a network that have lost synchronization. 

One possible technique is to require that each node provide an acknowledgment upon 
successful receipt of each packet and, if no acknowledgment is received within a certain period of 
time, to re-send the unacknowledged packet. This approach, however, drives up overhead costs 
and may be prohibitive in high-throughput environments such as streaming video or audio, for 
example. 

A different approach is to employ an automatic synchronizing technique that will be 
referred to herein as "self-synchronization." In this approach, synchronization information is 
embedded into each packet, thereby enabling the receiver to re-synchronize itself upon receipt of 
a single packet if it determines that is has lost synchronization with the sender. (If communications 
are already in progress, and the receiver determines that it is still in sync with the sender, then 
there is no need to re-synchronize.) A receiver could detect that it was out of synchronization by, 
for example, employing a "dead-man" timer that expires after a certain period of time, wherein the 
timer is reset with each valid packet. A time stamp could be hashed into the public sync field (see 
below) to preclude packet-retry attacks. 

In one embodiment, a "sync field" is added to the header of each packet sent out by the 
sender. This sync field could appear in the clear or as part of an encrypted portion of the packet. 
Assuming that a sender and receiver have selected a random-number generator (RNG) and seed 
value, this combination of RNG and seed can be used to generate a random-number sequence 
(RNS). The RNS is then used to generate a sequence of source/destination IP pairs (and, if 
desired, discriminator fields and hardware source and destination addresses), as described above. 
It is not necessary, however, to generate the entire sequence (or the first N-l values) in order to 
generate the Nth random number in the sequence; if the sequence index N is known, the random 
value corresponding to that index can be directly generated (see below). Different RNGs (and 
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seeds) with different fundamental periods could be used to generate the source and destination IP 
sequences, but the basic concepts would still apply. For the sake of simplicity, the following 
discussion will assume that IP source and destination address pairs (only) are hopped using a 
single RNG sequencing mechanism. 

In accordance with a "self-synchronization" feature, a sync field in each packet header 
provides an index (i.e., a sequence number) into the RNS that is being used to generate IP pairs. 
Plugging this index into the RNG that is being used to generate the RNS yields a specific random 
number value, which in turn yields a specific IP pair. That is, an IP pair can be generated directly 
from knowledge of the RNG, seed, and index number; it is not necessary, in this scheme, to 
generate the entire sequence of random numbers that precede the sequence value associated with 
the index number provided. 

Since the communicants have presumably previously exchanged RNGs and seeds, the only 
new information that must be provided in order to generate an IP pair is the sequence number. If 
this number is provided by the sender in the packet header, then the receiver need only plug this 
number into the RNG in order to generate an IP pair - and thus verify that the IP pair appearing 
in the header of the packet is valid. In this scheme, if the sender and receiver lose synchronization, 
the receiver can immediately re-synchronize upon receipt of a single packet by simply comparing 
the IP pair in the packet header to the IP pair generated from the index number. Thus, 
synchronized communications can be resumed upon receipt of a single packet, making this scheme 
ideal for multicast communications. Taken to the extreme, it could obviate the need for 
synchronization tables entirely; that is, the sender and receiver could simply rely on the index 
number in the sync field to validate the IP pair on each packet, and thereby eliminate the tables 
entirely. 

The aforementioned scheme may have some inherent security issues associated with it — 
namely, the placement of the sync field. If the field is placed in the outer header, then an interloper 
could observe the values of the field and their relationship to the IP stream. This could potentially 
compromise the algorithm that is being used to generate the IP-address sequence, which would 
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compromise the security of the communications. If, however, the value is placed in the inner 
header, then the sender must decrypt the inner header before it can extract the sync value and 
validate the IP pair; this opens up the receiver to certain types of denial-of-service (DoS) attacks, 
such as packet replay. That is, if the receiver must decrypt a packet before it can validate the IP 
pair, then it could potentially be forced to expend a significant amount of processing on 
decryption if an attacker simply retransmits previously valid packets. Other attack methodologies 
are possible in this scenario. 

A possible compromise between algorithm security and processing speed is to split up the 
sync value between an inner (encrypted) and outer (unencrypted) header. That is, if the sync 
value is sufficiently long, it could potentially be split into a rapidly-changing part that can be 
viewed in the clear, and a fixed (or very slowly changing) part that must be protected. The part 
that can be viewed in the clear will be called the "public sync" portion and the part that must be 
protected will be called the "private sync" portion. 

Both the public sync and private sync portions are needed to generate the complete sync 
value. The private portion, however, can be selected such that it is fixed or will change only 
occasionally. Thus, the private sync value can be stored by the recipient, thereby obviating the 
need to decrypt the header in order to retrieve it. If the sender and receiver have previously 
agreed upon the frequency with which the private part of the sync will change, then the receiver 
can selectively decrypt a single header in order to extract the new private sync if the 
communications gap that has led to lost synchronization has exceeded the lifetime of the previous 
private sync. This should not represent a burdensome amount of decryption, and thus should not 
open up the receiver to denial-of-service attack simply based on the need to occasionally decrypt 
a single header. 

One implementation of this is to use a hashing function with a one-to-one mapping to 
generate the private and public sync portions from the sync value. This implementation is shown 
in FIG. 13, where (for example) a first ISP 1302 is the sender and a second ISP 1303 is the 
receiver. (Other alternatives are possible from FIG. 13.) A transmitted packet comprises a public 
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or "outer" header 1305 that is not encrypted, and a private or "inner" header 1306 that is 
encrypted using for example a link key. Outer header 1305 includes a public sync portion while 
inner header 1306 contains the private sync portion. A receiving node decrypts the inner header 
using a decryption function 1307 in order to extract the private sync portion. This step is 
5 necessary only if the lifetime of the currently buffered private sync has (If the currently- 

buffered private sync is still valid, then it is simply extracted from memory and "added" (which 
could be an inverse hash) to the public sync, as shown in step 1308.) The public and decrypted 
private sync portions are combined in function 1308 in order to generate the combined sync 1309. 
The combined sync (1309) is then fed into the RNG (13 10) and compared to the IP address pair 
10 (13 1 1) to validate or reject the packet 
% An important consideration in this architecture is the concept of "future" and "past" where 

JE the public sync values are concerned. Though the sync values, themselves, should be random to 

JB prevent spoofing attacks, it may be important that the receiver be able to quickly identify a sync 

value that has already been sent — even if the packet containing that sync value was never 
W 15 actually received by the receiver. One solution is to hash a time stamp or sequence number into 
U the public sync portion, which could be quickly extracted, checked, and discarded, thereby 

21 validating the public sync portion itself. 

CI In one embodiment, packets can be checked by comparing the source/destination IP pair 

yfi generated by the sync field with the pair appearing in the packet header. If (1) they match, (2) the 

20 time stamp is valid, and (3) the dead-man timer has expired, then re-synchronization occurs; 
otherwise, the packet is rejected. If enough processing power is available, the dead-man timer and 
synchronization tables can be avoided altogether, and the receiver would simply resynchronize 
(e.g., validate) on every packet. 

The foregoing scheme may require large-integer (e.g., 160-bit) math, which may affect its 
25 implementation. Without such large-integer registers, processing throughput would be affected, 
thus potentially affecting security from a denial-of-service standpoint. Nevertheless, as large- 
integer math processing features become more prevalent, the costs of implementing such a feature 
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will be reduced. 

D. Other Synchronization Schemes 

As explained above, if W or more consecutive packets are lost between a transmitter and 
receiver in a YPN (where W is the window size), the receiver's window will not have been 
updated and the transmitter will be transmitting packets not in the receiver's window. The sender 
and receiver will not recover synchronization until perhaps the random pairs in the window are 
repeated by chance. Therefore, there is a need to keep a transmitter and receiver in 
synchronization whenever possible and to re-establish synchronization whenever it is lost. 

A "checkpoint" scheme can be used to regain synchronization between a sender and a 
receiver that have fellen out of synchronization. In this scheme, a checkpoint message comprising 
a random IP address pair is used for communicating synchronization information. In one 
embodiment, two messages are used to communicate synchronization information between a 
sender and a recipient: 

1. SYNC_REQ is a message used by the sender to indicate that it wants to synchronize; 
and 

2. SYNC_ACK is a message used by the receiver to inform the transmitter that it has 
been synchronized. 

According to one variation of this approach, both the transmitter and receiver maintain three 

checkpoints (see FIG. 14): 

L In the transmitter, ckpt_o ("checkpoint old") is the IP pair that was used to re-send the 
last SYNC_REQ packet to the receiver. In the receiver, ckpt o ("checkpoint old") is 
the IP pair that receives repeated SYNC_EEQ packets from the transmitter. 
2. In the transmitter, ckpt _n ("checkpoint new") is the IP pair that will be used to send 
the next SYNC_REQ packet to the receiver. In the receiver, ckptji ("checkpoint 
new") is the IP pair that receives a new SYNC_REQ packet from the transmitter and 
which causes the receiver's window to be re-aligned, ckpt_o set to ckptji, a new 
ckpt ji to be generated and a new ckpt_r to be generated. 
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3. In the transmitter, ckpt_r is the IP pair that will be used to send the next SYNC_ACK 
packet to the receiver. In the receiver, ckpt_r is the IP pair that receives a new 
SYNC_ACK packet from the transmitter and which causes a new ckpt_n to be 
generated. Since SYNC_ACK is transmitted from the receiver ISP to the sender ISP, 
the transmitter ckptj" refers to the ckptj: of the receiver and the receiver ckpt_r refers 
to the ckptj- of the transmitter (see FIG. 14). 
When a transmitter initiates synchronization, the IP pair it will use to transmit the next data packet 
is set to a predetermined value and when a receiver first receives a SYNC_REQ, the receiver 
window is updated to be centered on the transmitter's next IP pair. This is the primary mechanism 
for checkpoint synchronization. 

Synchronization can be initiated by a packet counter (e.g., after every N packets 
transmitted, initiate a synchronization) or by a timer (every S seconds, initiate a synchronization) 
or a combination of both. See FIG. 15. From the transmitter's perspective, this technique 
operates as follows: (1) Each transmitter periodically transmits a "sync request" message to the 
receiver to make sure that it is in sync. (2) If the receiver is still in sync, it sends back a "sync 
ack" message. (If this works, no further action is necessary). (3) If no "sync ack" has been 
received within a period of time, the transmitter retransmits the sync request again. If the 
transmitter reaches the next checkpoint without receiving a "sync ack" response, then 
synchronization is broken, and the transmitter should stop transmitting. The transmitter will 
continue to send sync_reqs until it receives a sync_ack , at which point transmission is 
reestablished. 

From the receiver's perspective, the scheme operates as follows: (1) when it receives a 
"sync request" request from the transmitter, it advances its window to the next checkpoint 
position (even skipping pairs if necessary), and sends a "sync ack" message to the transmitter. If 
sync was never lost, then the "jump ahead" really just advances to the next available pair of 
addresses in the table (i.e., normal advancement). 

If an interloper intercepts the "sync request" messages and tries to interfere with 



42 



0479.84602 



PATENT 



communication by sending new ones, it will be ignored if the synchronization has been established 
or it it will actually help to re-establish synchronization. 

A window is realigned whenever a re-synchronization occurs. This realignment entails 
updating the receiver's window to straddle the address pairs used by the packet transmitted 
immediately after the transmission of the SYNC_REQ packet. Normally, the transmitter and 
receiver are in synchronization with one another. However, when network events occur, the 
receiver's window may have to be advanced by many steps during ^synchronization. In this case, 
it is desirable to move the window ahead without having to step through the intervening random 
numbers sequentially. (This feature is also desirable for the auto-sync approach discussed above). 
E. Random Number Generator with a Jump-Ahead capability 

An attractive method for generating randomly hopped addresses is to use identical random 
number generators in the transmitter and receiver and advance them as packets are transmitted 
and received. There are many random number generation algorithms that could be used. Each one 
has strengths and weaknesses for address hopping applications. 

Linear congruential random number generators (LCRs) are fast, simple and well 
characterized random number generators that can be made to jump ahead n steps efficiently. An 
LCR generates random numbers Xi, X 2 , X 3 ... Xt starting with seed Xo using a recurrence 

Xf=(aXM + b)modc, (1) 
where a, b and c define a particular LCR. Another expression for Xi, 

XiKC^CXo+^-bVCa-l^modc (2) 
enables the jump-ahead capability. The factor a 1 can grow very large even for modest i if left 
unfettered. Therefore some special properties of the modulo operation can be used to control the 
size and processing time required to compute (2). (2) can be rewritten as: 

XKa ! (Xb(a-l>rt))-by(a-l) mode. (3) 
It can be shown that: 

(^(XoCa-O+^-bVCa-l) mod c = 

((a i mod((a-l)c)(Xi(a.l)rt)) -b) /(a-1)) mod c (4). 
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(Xo(a-l)+b) can be stored as (Xo(a-l)+b) mod c, b as b mod c and compute a 1 mod((a-l)c) (this 
requires 0(log(i)) steps). 

A practical implementation of this algorithm would jump a fixed distance, n, between 
synchronizations; this is tantamount to synchronizing every n packets. The window would 
commence n IP pairs from the start of the previous window. Using Xj w , the random number at the 
j* checkpoint, as Xo and n as /, a node can store a n mod((a-l)c) once per LCR and set 

Xj+i^Xno^^mod^a-Oc) (Xj w (a-l)+b)-b)/(a-l))mod c, (5) 
to generate the random number for the j+l* synchronization. Using this construction, a node 
could jump ahead an arbitrary (but fixed) distance between synchronizations in a constant amount 
of time (independent of n). 

Pseudo-random number generators, in general, and LCRs, in particular, will eventually 
repeat their cycles. This repetition may present vulnerability in the IP hopping scheme. An 
adversary would simply have to wait for a repeat to predict fiiture sequences. One way of coping 
with this vulnerability is to create a random number generator with a known long cycle. A random 
sequence can be replaced by a new random number generator before it repeats. LCRs can be 
constructed with known long cycles. This is not currently true of many random number 
generators. 

Random number generators can be cryptographically insecure. An adversary can derive 
the RNG parameters by examining the output or part of the output. This is true of LCGs. This 
vulnerability can be mitigated by incorporating an encryptor, designed to scramble the output as 
part of the random number generator. The random number generator prevents an adversary from 
mounting an attack — e.g., a known plaintext attack — against the encryptor. 
F. Random Number Generator Example 

Consider a RNG where a=3 l,b=4 and c=15. For this case equation (1) becomes: 

Xi=(31XM + 4)modl5. (6) 

If one sets Xo=l, equation (6) will produce the sequence 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 1 1, 
0, 4, 8, 12. This sequence will repeat indefinitely. For a jump ahead of 3 numbers in this sequence 
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a »= 31 3 =29791, c*(a-l)=15*30=450 and a" mod((a-l)c) = 31 3 mod(15*30)=29791mod(450)=91. 
Equation (5) becomes: 

((91 (Xi30+4)-4)/30)mod 15 (7). 
Table 1 shows the jump ahead calculations from (7) . The calculations start at 5 and jump ahead 
3. 
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TABLE 1 



I 


Xi 


(Xi30+4) 


91 (Xi30+4)-4 


((91 (Xi30+4)-4)/30 


Xj+3 


1 


5 


154 


14010 


467 


2 


4 


2 


64 


5820 


194 


14 


7 


14 


424 


38580 


1286 


11 


10 


11 


334 


30390 


1013 


8 


13 


8 


244 


22200 


740 


5 



G. Fast Packet Filter 

Address hopping VPNs must rapidly determine whether a packet has a valid header and 
thus requires further processing, or has an invalid header (a hostile packet) and should be 
immediately rejected. Such rapid determinations will be referred to as "fast packet filtering," This 
capability protects the VPN from attacks by an adversary who streams hostile packets at the 
receiver at a high rate of speed in the hope of saturating the receiver's processor (a so-called 
"denial of service" attack). Fast packet filtering is an important feature for implementing VPNs on 
shared media such as Ethernet 

Assuming that all participants in a VPN share an unassigned "A" block of addresses, one 
possibility is to use an experimental "A" block that will never be assigned to any machine that is 
not address hopping on the shared medium. "A" blocks have a 24 bits of address that can be 
hopped as opposed to the 8 bits in "C" blocks. In this case a hopblock will be the "A" block. The 
use of the experimental "A" block is a likely option on an Ethernet because: 

1. The addresses have no validity outside of the Ethernet and will not be routed out to a valid 
outside destination by a gateway. 

2. There are 2 24 (-16 million) addresses that can be hopped within each "A" block. This yields 
>280 trillion possible address pairs making it very unlikely that an adversary would guess a 
valid address. It also provides acceptably low probability of collision between separate VPNs 
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(all VPNs on a shared medium independently generate random address pairs from the same 
"A" block). 

3 . The packets will not be received by someone on the Ethernet who is not on a VPN (unless the 
machine is in promiscuous mode) minimizing impact on non-VPN computers. 

The Ethernet example will be used to describe one implementation of fast packet filtering. 
The ideal algorithm would quickly examine a packet header, determine whether the packet is 
hostile, and reject any hostile packets or determine which active IP pair the packet header 
matches. The problem is a classical associative memory problem. A variety of techniques have 
been developed to solve this problem (hashing, B-trees etc). Each of these approaches has its 
strengths and weaknesses. For instance, hash tables can be made to operate quite fast in a 
statistical sense, but can occasionally degenerate into a much slower algorithm. This slowness can 
persist for a period of time. Since there is a need to discard hostile packets quickly at all times, 
hashing would be unacceptable. 
H. Presence Vector Algorithm 

A presence vector is a bit vector of length 2 tt that can be indexed by w-bit numbers (each 
ranging from 0 to 2 tt -l). One can indicate the presence of k n-bit numbers (not necessarily 
unique), by setting the bits in the presence vector indexed by each number to 1. Otherwise, the 
bits in the presence vector are 0. An w-bit number, x, is one of the k numbers if and only if the X th 
bit of the presence vector is 1. A fast packet filter can be implemented by indexing the presence 
vector and looking for a 1, which will be referred to as the "test." 

For example, suppose one wanted to represent the number 135 using a presence vector. 
The DS^bit of the vector would be set. Consequently, one could very quickly determine whether 
an address of 135 was valid by checking only one bit: the 135 th bit The presence vectors could be 
created in advance corresponding to the table entries for the IP addresses. In effect, the incoming 
addresses can be used as indices into a long vector, making comparisons very fast. As each RNG 
generates a new address, the presence vector is updated to reflect the information. As the 
window moves, the presence vector is updated to zero out addresses that are no longer valid. 
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There is a trade-off between efficiency of the test and the amount of memory required for 
storing the presence vector(s). For instance, if one were to use the 48 bits of hopping addresses as 
an index, the presence vector would have to be 35 terabytes. Clearly, this is too large for practical 
purposes. Instead, the 48 bits can be divided into several smaller fields. For instance, one could 
5 subdivide the 48 bits into four 12-bit fields (see FIG. 16). This reduces the storage requirement to 
2048 bytes at the expense of occasionally having to process a hostile packet. In effect, instead of 
one long presence vector, the decomposed address portions must match all four shorter presence 
vectors before further processing is allowed. (If the first part of the address portion doesn't 
match the first presence vector, there is no need to check the remaining three presence vectors). 
10 A presence vector will have a 1 in the y & bit if and only if one or more addresses with a 

yfi corresponding field of y are active. An address is active only if each presence vector indexed by 

31 the appropriate sub-field of the address is 1 . 

jO Consider a window of 32 active addresses and 3 checkpoints. A hostile packet will be 

jp rejected by the indexing of one presence vector more than 99% of the time. A hostile packet will 

^ 15 be rejected by the indexing of all 4 presence vectors more than 99.9999995% of the time. On 

average, hostile packets will be rejected in less than 1.02 presence vector index operations. 
Rj The small percentage of hostile packets that pass the fast packet filter will be rejected 

% when matching pairs are not found in the active window or are active checkpoints. Hostile 

* packets that serendipitously match a header will be rejected when the VPN software attempts to 

20 decrypt the header. However, these cases will be extremely rare. There are many other ways this 

method can be configured to arbitrate the space/speed tradeoffs. 

L Further Synchronization Enhancements 

A slightly modified form of the synchronization techniques described above can be 

employed. The basic principles of the previously described checkpoint synchronization scheme 
25 remain unchanged. The actions resulting from the reception of the checkpoints are, however, 

slightly different. In this variation, the receiver will maintain between OoO ("Out of Order") and 

2xWINDOW_SIZE+OoO active addresses (1 <OoO <WINDOW_SIZE and WINDOW_SIZE 
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£l). OoO and WINDOWJSIZE are engineerable parameters, where OoO is the minimum number 
of addresses needed to accommodate lost packets due to events in the network or out of order 
arrivals and WINDOW_SIZE is the number of packets transmitted before a SYNC_REQ is 
issued. FIG. 17 depicts a storage array for a receiver's active addresses. 

The receiver starts with the first 2xWINDOWjSIZE addresses loaded and active 
(ready to receive data). As packets are received, the corresponding entries are marked as "used" 
and are no longer eligible to receive packets. The transmitter maintains a packet counter, initially 
set to 0, containing the number of data packets transmitted since the last initial transmission of a 
SYNCJREQ for which SYNC_ACK has been received. When the transmitter packet counter 
equals WINDOWJSIZE, the transmitter generates a SYNC_REQ and does its initial transmission. 
When the receiver receives a SYNC_REQ corresponding to its current CKPTN, it generates the 
next WINDOWJSIZE addresses and starts loading them in order starting at the first location after 
the last active address wrapping around to the beginning of the array after the end of the array has 
been reached. The receiver's array might look like FIG. 18 when a SYNC_REQ has been 
received. In this case a couple of packets have been either lost or will be received out of order 
when the SYNCREQ is received. 

FIG. 19 shows the receiver's array after the new addresses have been generated. If the 
transmitter does not receive a SYNC_ACK, it will re-issue the SYNC_REQ at regular intervals. 
When the transmitter receives a SYNC_ACK, the packet counter is decremented by 
WINDOW_SIZE. If the packet counter reaches 2xWINDOW_SIZE - OoO then the transmitter 
ceases sending data packets until the appropriate SYNC_ACK is finally received. The transmitter 
then resumes sending data packets. Future behavior is essentially a repetition of this initial cycle. 
The advantages of this approach are: 

1 . There is no need for an efficient jump ahead in the random number generator, 

2. No packet is ever transmitted that does not have a corresponding entry in the receiver side 

3. No timer based re-synchronization is necessary. This is a consequence of 2. 

4. The receiver will always have the ability to accept data messages transmitted within OoO 
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messages of the most recently transmitted message. 
JL Distributed Transmission Path Variant 

Another embodiment incorporating various inventive principles is shown in FIG. 20. In 
this embodiment, a message transmission system includes a first computer 2001 in communication 
with a second computer 2002 through a network 201 1 of intermediary computers. In one variant 
of this embodiment, the network includes two edge routers 2003 and 2004 each of which is linked 
to a plurality of Internet Service Providers (ISPs) 2005 through 2010. Each ISP is coupled to a 
plurality of other ISPs in an arrangement as shown in FIG. 20, which is a representative 
configuration only and is not intended to be limiting. Each connection between ISPs is labeled in 
FIG. 20 to indicate a specific physical transmission path (e.g., AD is a physical path that links ISP 
A (element 2005) to ISP D (element 2008)). Packets arriving at each edge router are selectively 
transmitted to one of the ISPs to which the router is attached on the basis of a randomly or quasi- 
randomly selected basis. 

As shown in FIG. 21, computer 2001 or edge router 2003 incorporates a plurality of link 
transmission tables 2100 that identify, for each potential transmission path through the network, 
valid sets of IP addresses that can be used to transmit the packet. For example, AD table 2101 
contains a plurality of IP source/destination pairs that are randomly or quasi-randomly generated. 
When a packet is to be transmitted from first computer 2001 to second computer 2002, one of the 
link tables is randomly (or quasi-randomly) selected, and the next valid source/destination address 
pair from that table is used to transmit the packet through the network. If path AD is randomly 
selected, for example, the next source/destination IP address pair (which is pre-determined to 
transmit between ISP A (element 2005) and ISP B (element 2008)) is used to transmit the packet. 
If one of the transmission paths becomes degraded or inoperative, that link table can be set to a 
"down" condition as shown in table 2105, thus preventing addresses from being selected from 
that table. Other transmission paths would be unaffected by this broken link. 
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CLAIMS 

1 . A method of transmitting information between a first computer and a second computer, 
comprising the steps of: 

(1) embedding in each of a plurality of data packets a discriminator value that periodically 
5 changes between successive data packets, wherein each discriminator value is not based solely on 

the value of other data in each data packet; 

(2) transmitting the plurality of data packets between the first computer and the second 
computer; 

(3) receiving the transmitted data packets at the second computer; and 

10 (4) for each received data packet, comparing the discriminator value to a set of valid 

discriminator values and, in response to detecting a match, accepting the received data packet for 
further processing, and otherwise rejecting the received data packet 

2. The method of claim 1, wherein step (1) comprises the step of using an Internet 
Protocol address in an Int^roetProtocol header as the discriminator value, wherein the Internet 

15 Protocol address is used to route the data packets over the Internet. 

3. The method of claim 2, further comprising the step of changing in value only part of 
the Internet Protocol addressesTietween successive packets. 

4. The method of claim 1, further comprising the step of using as the discriminator value a 
data field external to an InternefProtocol header of each data packet. 

20 5. The method of claim 1, wherein steps (1) and (4) are performed in a data link layer of 

an ISO standard communicationprotocol. 

6. The method of claim 1, wherein step (1) comprises the step of using a Media Access 
Control (MAC) hardware addresslsthe discriminator value, wherein the MAC hardware address 
is used to route the data packets on a local area network. 
25 7. The method of claim 1, wherein step (1) comprises the step of using a different 

discriminator value for each successive packet. 

8. The method of claim 1, wherein step (4) comprises the step of comparing each 
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discriminator value to a window of valid discriminator values, wherein the window is wide 
enough to permit comparison to only a small number of potentially valid discriminator values, and 
further comprising the step of moving the window as successive data packets are received. 

9. The method of claim 1, further comprising the step of sharing between the first 
computer and the second comguter information sufficient to generate the set of valid 
discriminator values. 

10. The method of claim 1, further comprising the step of transmitting from the first 
computer to the second computermalgorithm for selecting successively valid discriminator 
values. 

11. The method of claim 1, wherein step (4) comprises the step of using a presence vector 
to determine whether to accepFBach data packet. 

12. The method of claim 1, wherein step (4) comprises the step of using a hashing 
function to determine whether thediscriminator value is valid. 

13. The method of claim 1, further comprising the step of transmitting a synchronization 
request between the first computer and the second computer, wherein the second computer uses 
the synchronization request to maintain synchronization of valid discriminator values. 

14. The method of claim 13, further comprising the step of, in response to failure to 
receive a synchronization ackiwwTedgement from the second computer, shutting off transmission 
of data packets to the second computer. 

15. The method of claim 13, further comprising the step of embedding a synchronization 
value in each data packet that permits the second computer to re-establish synchronization in a set 
of potentially valid discriminator values. 

16. The method of claim 13, further comprising the step of moving a window of valid 
discriminator values in the second computer in response to receiving the synchronization request 
from the first computer. ^^-^ 

17. The method of claim 1, wherein step (1) comprises the steps of using an Internet 
Protocol source address in an Internet Protocol header as a first part of the discriminator value 
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and using an Internet Protocol destination address in the Internet Protocol header as a second part 
of the discriminator value, wherein the source and destination addresses are used to route each 
data packet over the Internet. 

18. The method of claim 17, further comprising the steps of: 
embedding a plurality of thedata packets into a frame; and 

embedding a source and destination hardware address in the frame, wherein the source 
and destination hardware address are quasi-randomly generated and used to route the frame on a 
network. 

19. The method of claim 1, further comprising the step of maintaining in the first 
computer a first transmit table^antf^first receive table, and maintaining in the second computer a 
second transmit table and a second receive table, 

wherein each transmit table comprises a list of valid discriminator values that are to be 
inserted into outgoing data packets; 

wherein each receive table comprises a list of valid discriminator values that are to be 
compared against incoming data packets; and 

wherein the first transmit table in the first computer matches the second receive table in 
the second computer, and wherein the first receive table in the first computer matches the second 
transmit table in the second computer. 

20. A method of transmitting data packets over a network comprising a plurality of 
computers connected to each other through a plurality of physical transmission paths, the method 
comprising the steps of: 

(1) for each of the plurality of data packets, randomly selecting one of the plurality of 
physical transmissions paths through the plurality of computers; and 

(2) transmitting each data packet over the randomly selected physical transmission path. 

21. The method of claim 20, wherein step (1) comprises the steps of: 

(a) selecting a path defined byli pair of computers in the network; 

(b) selecting valid source and destination addresses associated with the selected path; and 
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(c) inserting the valid source and destination addresses into the data packet before 
transmitting it over the selected path. 

22. The method of claim 21, wherein step (1) comprises the step of avoiding selection of 
a path that is not operational. 

23. A system comprising: 

a first computer that embeds into each of a plurality of data packets a discriminator value 
that periodically changes between successive data packets, wherein each discriminator value is not 
based solely on the value of other data in each data packet; and 

a second computer coupled to the first computer through a network, 

wherein the first computer transmits the plurality of data packets to the second computer, 

and 

wherein the second computer receives the transmitted data packets, compares the 
discriminator value in each received data packet to a set of valid discriminator values and, in 
response to detecting a match, accepts the received data packet for further processing, and 
otherwise rejects the received data packet. 

24. The system of claim 23, wherein the first computer embeds into each of the plurality 
of data packets an Internet Protocdaddress in an Internet Protocol header as the discriminator 
value, wherein the Internet Protocol address is used to route the data packets over the Internet. 

25. The system of claim 24, wherein the first computer changes in value only part of the 
Internet Protocol addresses betweenrsuccessive packets. 

26. The system of claim 23, wherein the first computer embeds the discriminator value in 
a data field external to an Intern^tftotocol header of each data packet. 

27. The system of claim 23, wherein the first computer embeds each discriminator value 
in a first data link layer of an ISO-standard communication protocol, and wherein the second 
computer compares each discriminator value in a second data link layer of the ISO standard 
communications protocol. 
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28. The system of claim 23, wherein the first computer embeds a Media Access Control 
(MAC) hardware address as the-dtscriminator value, wherein the MAC hardware address is used 
to route the data packets on a local area network. 

29. The system of claim 23, wherein the first computer embeds a different discriminator 
value for each successive packets 

30. The system of claim 23, wherein the second computer compares each discriminator 
value to a window of valid discriminator values, wherein the window is wide enough to permit 
comparison to only a small number of potentially valid discriminator values, and wherein the 
window is moved as successive data packets are received. 

31. The system of claim 23, wherein the first and second computers share common 
information sufficient to generatgiheset of valid discriminator values. 

32. The system of claim 23, wherein the first computer transmits to the second computer 
an algorithm for selecting successively valid discriminator values. 

33 . The system of claim 23, wherein the second computer uses a presence vector to 
determine whether to accept eadi&rta packet. 

34. The system of claim 23, wherein the second computer uses a hashing fiinction to 
determine whether the (tiscriminatorvalue is valid. 

35. The system of claim 23, wherein the first computer transmits to the second computer 
a synchronization request, whereuuthe second computer uses the synchronization request to 
maintain synchronization of valid discriminator values. 

36. The system of claim 35, wherein the first computer, in response to Mure to receive a 
synchronization acknowledgementfrom the second computer, shuts off transmission of data 
packets to the second computer. 

37. The system of claim 35, wherein the first computer embeds a synchronization value in 
each data packet that permits the^econd computer to re-establish synchronization in a set of 
potentially valid discriminator values. 
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38. The system of claim 35, wherein the second computer moves a window of valid 
discriminator values in responseie-receiving the synchronization request from the first computer. 

39. The system of claim 23, wherein the first computer embeds an Internet Protocol 
source address in an Internet Protocol header as a first part of the discriminator value and embeds 
an Internet Protocol destination address in the Internet Protocol header as a second part of the 
discriminator value, wherein the source and destination addresses are used to route each data 
packet over the Internet. 

40. The system of claim 39, wherein the first computer embeds a plurality of the data 
packets into a frame and embedsTsource and destination hardware address in the frame, wherein 
the source and destination hardware address are quasi-randomly generated and used to route the 
frame on a network. 

41. The system of claim 23, 

wherein the first computerc^mprises a first transmit table and a first receive table, 
wherein the second computer comprises a second transmit table and a second receive 

table, 

wherein each transmit table comprises a list of valid discriminator values that are to be 
inserted into outgoing data packets, 

wherein each receive table comprises a list of valid discriminator values that are to be 
compared against incoming data packets, 

wherein the first transmit table in the first computer matches the second receive table in 
the second computer, and 

wherein the first receive table in the first computer matches the second transmit table in 
the second computer. 

42. A first computer coupled to a network comprising a plurality of computers connected 
to each other through a plurality of physical transmission paths, 

wherein the first computer generates a plurality of data packets for transmission acrossthe 
network; and 
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wherein the first computer, for each of the plurality of data packets, randomly selects one 
of the plurality of physical transmissions paths through the plurality of computers and transmits 
each data packet over the randomly selected physical transmission path. 

43. The first computer of claim 42, wherein the first computer: 

(a) selects a path defined by a pair of computers in the network; 

(b) selects valid source and destination addresses associated with the selected path; and 

(c) inserts the valid source and destination addresses into the data packet before 
transmitting it over the selected path. 

44. The first of claim 43, wherein the first computer avoids the step of avoiding selection 
of a path that is not operatiofialr\ 

45. A system comprising in combination: 

a transmitting node that generates pseudo-random discriminator values and embeds the 
pseudo-random discriminator values into data packets for transmission; and 

a receiving node that receives data packets transmitted by the transmitting node, wherein 
the receiving node, for each received packet, extracts the pseudo-randomly generated 
discriminator value, compares it to a set of potentially valid discriminator values shared between 
the transmitting node and the receiving node and, in response to detecting a match, accepts the 
data packet, and otherwise discards the packet. 

46. The system of claim 45, wherein the receiving node maintains a window of valid 
discriminator values, wherein the vtfndow is moved in response to detecting a match. 

47. The system of claim 45, wherein each pseudo-randomly generated discriminator value 
comprises a valid Internet Protocol address that is assigned to the receiving node. 

48. The system of claim 45, wherein each pseudo-randomly generated discriminator value 
comprises a valid Media AccesslDontrol (MAC) hardware address that is assigned to the 
receiving node. 

49. The system of claim 45, wherein the transmitting node generates a different pseudo- 
randomly generated discriminator value for each successive data packet. 
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50. A receiving computer that receives data packets from a transmitting computer, 
wherein the receiving computer comprises computer instructions that execute the steps of: 

(1) for each received data packet, extracting a discriminator value inserted by the 
transmitting computer; 

(2) comparing the extracted discriminator value to a set of valid discriminator values on 
the basis of information previously shared with the transmitting computer; and 

(3) in response to detecting a match in step (2), accepting the received data packet for 
further processing and otherwise rejecting the data packet. 

51. The receiving computer of claim 50, wherein the receiving computer further 
comprises computer instructions that exfcragfasthe discriminator value an Internet Protocol 
address from a header portion of each data packet 

52. The receiving computer of claim 50, wherein the receiving computer maintains a 
window of valid discriminator values, wherein the window is moved in response to detecting 
matches. 

53. The receiving computer of claim 50, wherein the receiving computer receives 
information from the transmitting computer sufficient to establish the set of valid discriminator 
values. 

54. A method of transmitting data from a first computer to a second computer, the data 
comprising a plurality of data bytes arranged in a particular order, the method comprising the 
steps of: 

(1) establishing in the first computer and second computer a common algorithm that 
determines how data will be randomly distributed across a plurality of data packets; 

(2) in the first computer, randomly distributing the plurality of data bytes across the 
plurality of data packets according to the common algorithm; 

(3) transmitting the plurality of data packets from the first computer to the second 
computer; and 

(4) in the second computer, extracting the randomly distributed plurality of data bytes 
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from the plurality of data packets and reassembling them into the particular order according to the 



common algorithm. 

55. The method of cjahn 1, wherein step (3) comprises the step of transmitting each of 
the plurality of data packets across a different path in a computer network. 

56. A system comprising: 

a first computer including an algorithm that establishes a random distribution pattern for 
allocating data across a plurality of data packets, wherein the first computer randomly distributes 
data bytes from a data source across the plurality of data packets according to the random 
distribution pattern and transmits the plurality of data packets across a network; and 

a second computer coupled to the first computer across the network, wherein the second 
computer receives the plurality of data packets from the first computer, extracts the randomly 
distributed data bytes, and reassembles them into their original order according to the algorithm. 

57. The system of claim 56, wherein the first computer transmits each of the plurality of 
data packets across a different path in the network. 

58. A method of securely transmitting a data packet between a sending computer and a 
receiving computer, comprising the steps of: 

(1) encrypting the data packet using a session key known to the sending computer and the 
receiving computer, but not known by intermediate computers between the sending computer and 
the receiving computer; 

(2) adding a packet header that identifies the data packet to the data packet encrypted in 
step(l); 

(3) encrypting the combined packet header and encrypted data packet created in step (2) 
using a link key known to each of a plurality of intermediate computers arranged between the first 
computer and the second computer; 

(4) adding a cleartext packet header to route the packet encrypted in step (3); and 

(5) transmitting the packet created in step (4). 

59. The method of claim 58, fiirther comprising the steps of: 
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(5) at each intermediate computer, decrypting the packet received from a previous 
computer and decrypting it using the link key; 

(6) re-encrypting the packet using a different link key known to a next intermediate 
computer in the network; 

(7) adding a cleartext packet header to route the packet re-encrypted in step (6); and 

(8) transmitting the packet created in step (7) to the next intermediate computer. 

60. The method of claini59, further comprising the step of, at the receiving computer, 
decrypting the packet using the session key. 

61 . A method of transmitting data over a computer network, comprising the steps of: 

at an originating terminal connected to the computer network, receiving a stream of data 
and forming first level data packet payloads therefrom; 

identifying a network destination address for the stream of data and adding first level 
headers containing data representing the network destination address to each of the data packets 
to form a first level packet; 

encrypting each of the first level packets to form second level packet payloads; 
attaching to the second level packet, payloads headers containing as destination addresses, 
addresses of at least one intermediate router connecting the originating terminal to the destination 
to form second level packets; 

sending the second level packets to the at least one intermediate router; 

at the at least one intermediate router, decrypting at least one of the second level payloads 
and determining from the first level headers the destination address, forming new packets 
containing at least the first level packet payloads, and attaching headers thereto containing the 
destination address, whereby a true destination of the data stream is concealed behind a layer of 
encryption for at least a portion of its travel over the network. 

62. The method of claim 61, wherein the step of attaching includes determining the at least 
one intermediate router by randomly selecting from a group of intermediate routers. 

63. The method of claim 61, wherein the step of determining from the first level headers 
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the destination address includes converting the data representing the network destination address 
with the network destination address by means of correlation data stored on the intermediate 
router. 

64. The method of claim 61, further comprising the step of including in one of the first 
5 and second layer headers, an indicator of a number of hops to be made by the first level packet 

before arriving at the network destination, the at least one intermediate router decrementing the 
indicator of a number of hops and sending the first level packet to another intermediate router 
responsively to a value of the indicator of a number of hops. 

65. A method of routing packets on a packet network, comprising the steps of: \^ 
10 block-encrypting, with a session key, message data to form payloads; 

dividing an encrypted block resulting from the block-encrypting into at least two data 
payloads such that interleaving portions of data resulting from the block-encrypting step are 
among the at least two data payloads; 

encrypting, with a link key, each of the at least two data payloads, together with 
15 destination data identifying a final destination for the packets; 

combining, with a first payload resulting from the last step of encrypting, a first hop 
address indicating a first intermediate destination address and transmitting a first packet resulting 
thereby to the first intermediate destination address; 

combining, with a second payload resulting from the last step of encrypting, a second hop 
20 address indicating a second intermediate destination address and transmitting a second packet 
resulting thereby to the second intermediate destination address. 

66. The method of claim 65, further comprising the steps of: 
combining, in the first packet^ first hop counter; 

at a terminal coinciding with the first intermediate destination address, determining, 
25 responsively to the first hop counter, to send the first packet to the final destination address; and 
at the terminal coinciding with the first intermediate destination address, decrypting with 
the link key the first payload to expose the final destination address and sending the first packet to 
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the final destination address, responsively to the step of determining. 
67. The method of claim 65, further comprising the steps of: 
combining, in the second packet, a second hop counter; 

at a terminal coinciding with the second intermediate destination address, determining, 
responsively to the second hop counter, to send the first packet to the final destination address; 

at the terminal coinciding with the second intermediate destination address, decrypting 
with the link key the second payload to expose the final destination address and sending the 
second packet to the final destination address, responsively to the last step of determining. 
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ABSTRACT 

A plurality of computer nodes communicates using seemingly random IP source and 
destination addresses and (optionally) a seemingly random discriminator field. Data packets 
matching criteria defined by a moving window of valid addresses are accepted for further 
processing, while those that do not meet the criteria are rejected. In addition to "hopping" of IP 
addresses and discriminator fields, hardware addresses such as Media Access Control addresses 
can be hopped. The hopped addresses are generated by random number generators having non- 
repeating sequence lengths that are easily determined a-priori, which can quickly jump ahead in 
sequence by an arbitrary number of random steps and which have the property that future random 
numbers are difficult to guess without knowing the random number generator's parameters. 
Synchronization techniques can be used to re-establish synchronization between sending and 
receiving nodes. These techniques include a self-synchronization technique in which a sync field 
is transmitted as part of each packet, and a "checkpoint" scheme by which transmitting and 
receiving nodes can advance to a known point in their hopping schemes. A fast-packet reject 
technique based on the use of presence vectors is also described. A distributed transmission path 
embodiment incorporates randomly selected physical transmission paths. 
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